Symptom checker apps are often unreliable

Recent years have seen the emergence of health apps, which make a diagnosis based on symptoms and health complaints that patients have entered into the app. These apps are called symptom checkers. However, research carried out by the Harvard Medical School shows the majority of English-language symptom checkers are unreliable.

How do symptom checkers work?

After the symptoms have been entered into the app, symptom checkers either make a diagnosis (or group of likely diagnoses) or indicate an urgency. This urgency can be divided into three groups: 1. Immediately consult a doctor 2. Make an appointment with a doctor within a couple of days. 3. Self-care is sufficient.

Research outline

A research group from the Harvard Medical School has tested the reliability of 23 English-language symptom checkers. They published their results in the British Medical Journal. The study can be downloaded here. The researchers used symptoms and complaints that pointed to specific diagnoses, according to professional standards. They used 45 different examples of paper patients or vignettes: fifteen per urgency group as mentioned above. Ultimately, the researchers used 770 vignettes to assess the diagnosis made by the 23 checkers and 550 to assess the urgency determination made by these apps. Some symptom checkers did both with the same vignette.


For 34% of the 77 vignettes, the checkers arrived at the correct diagnosis. For 58% the correct diagnosis belonged to the group of the first twenty probable diagnoses. For 57% of the aforementioned 550 urgency vignettes, the symptom checkers placed the patients in the correct urgency group. For immediate activation of professional help (group 1), this percentage amounted to 80% and for the second group it amounted to 55%. For two thirds of the vignettes for whom self-care was sufficient according to professional standards (group 3), the symptom checkers nonetheless recommended professional care.

Dutch versions

So far the results of this study. In the Netherlands, symptom checkers are also being used. Some widely-used apps include the Pocket Doctor (de Dokter-op-zak), created by healthcare insurer FBTO, and the app Do I have to go to the doctor? ( . In addition, there are numerous apps for specific diagnostic areas such as cancer, mental healthcare and chronic care, which answer questions about symptoms. Whether these Dutch symptom checkers are more reliable than the English versions tested here has never been researched. This is a task for patient organisations, the Dutch consumer association, health insurance companies and the Dutch Health Care Inspectorate.


I found this article during the process of writing my book Integrated Care: Better and Cheaper which is published by Reed Elsevier. You can buy it here.

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>