In recent years, social media has become a rich source of mental health data. However, there is a lack of web-based research on the accuracy and validity of self-reported diagnostic information available on the web.
An analysis of the degree of correspondence between self-reported diagnoses and clinical indicators will afford researchers and clinicians higher levels of trust in social media analyses. We hypothesized that self-reported diagnoses would correspond to validated disorder-specific severity questionnaires across 2 large web-based samples.
The participants of study 1 were 1123 adults from a national Qualtrics panel (mean age 34.65, SD 12.56 years; n=635, 56.65% female participants,). The participants of study 2 were 2237 college students from a large university in the Midwest (mean age 19.08, SD 2.75 years; n=1761, 75.35% female participants). All participants completed a web-based survey on their mental health, social media use, and demographic information. Additionally, the participants reported whether they had ever been diagnosed with a series of disorders, with the option of selecting “Yes”; “No, but I should be”; “I don’t know”; or “No” for each condition. We conducted a series of ANOVA tests to determine whether there were differences among the 4 diagnostic groups and used post hoc Tukey tests to examine the nature of the differences.
In study 1, for self-reported mania (F=2.75; P=.04), somatic symptom disorder (F=26.75; P<.001), and alcohol use disorder (F=77.73; P<.001), the pattern of mean differences did not suggest that the individuals were accurate in their self-diagnoses. In study 2, for all disorders but bipolar disorder (F,=1.43; P=.23), ANOVA results were consistent with our expectations. Across both studies and for most conditions assessed, the individuals who said that they had been diagnosed with a disorder had the highest severity scores on self-report questionnaires, but this was closely followed by individuals who had not been diagnosed but believed that they should be diagnosed. This was especially true for depression, generalized anxiety, and insomnia. For mania and bipolar disorder, the questionnaire scores did not differentiate individuals who had been diagnosed from those who had not.
In general, if an individual believes that they should be diagnosed with an internalizing disorder, they are experiencing a degree of psychopathology similar to those who have already been diagnosed. Self-reported diagnoses correspond well with symptom severity on a continuum and can be trusted as clinical indicators, especially in common internalizing disorders such as depression and generalized anxiety disorder. Researchers can put more faith into patient self-reports, including those in web-based experiments such as social media posts, when individuals report diagnoses of depression and anxiety disorders. However, replication and further study are recommended.

©Lauren A Rutter, Jacqueline Howard, Prabhvir Lakhan, Danny Valdez, Johan Bollen, Lorenzo Lorenzo-Luaces. Originally published in JMIR Formative Research (https://formative.jmir.org), 13.01.2023.