Automatic diagnosis of mental healthcare information actionability: Developing binary classifiers

We aimed to develop a quantitative instrument to assist with the automatic evaluation of the actionability of mental healthcare information. We collected and classified two large sets of mental health information from certified mental health websites: generic and patient-specific mental healthcare information. We compared the performance of the optimised classifier with popular readability tools and non-optimised classifiers in predicting mental health information of high actionability for people with mental disorders. sensitivity of the classifier using both semantic and structural features as variables achieved statistically higher than that of the binary classifier using either semantic (p < 0.001) or structural features (p = 0.0010). The specificity of the optimized classifier was statistically higher than that of the classifier using structural variables (p = 0.002) and the classifier using semantic variables (p = 0.001). Differences in specificity between the full-variable classifier and the optimised classifier were statistically insignificant (p = 0.687). These findings suggest the optimised classifier using as few as 19 semantic-structural variables was the best-performing classifier. By combining insights of linguistics and statistical analyses, we effectively increased the interpretability and the diagnostic utility of the binary classifiers to guide the development, evaluation of the actionability and usability of mental healthcare information.