Covid-19: Are medical chatbots useful?

medical chatbot

One of the many benefits of Digital Healthcare is that medicine is that the patient is now claiming back his position at the center of his medical journey. It has never been so easy to collect information that would have required to compile knowledge of all of the hospital staff just twenty years ago. Thanks to the Internet, medicine is now efficient, meaning that all information is freely available. However, this babel library’s main drawback is that knowledge is unfiltered and subject to the reader’s interpretation.


Medical chatbots have tremendous potential for helping health services, whether in the United States or globally. They offer unique benefits that can benefit the patients, the practitioners, and the healthcare industry.

1. Availability

When patients have a question to ask or feel stressed, medical chatbots will diligently provide health information whatever the day and whatever the time. In itself, chatbots can not be compared to practitioners who might not find the time to answer often simple questions for which patients are anxiously looking for answers. Chatbots have the potential to streamline healthcare services and act as the first line of triage. 

2. Providing additional information

Patients often feel shy when in front of their doctor or may not think about a question that they may have. Chatbots can be seen as customer services that will be here to help provide additional guidance when needed.


Many people are seeking information when dealing with the current pandemic. Checking the Center for Disease Control website takes time, and the amount of information provided may be overwhelming. 

Medical chatbots powered by artificial intelligence are now an integral part of how people access medical general information. The Internet is full of medical websites ready to provide answers to any health-related question. But, are the answers provided by these chatbots accurate and reliable enough? This question is critical in evaluating if they can be used safely by people in need of answers.

Especially for the COVID-19 pandemic, most telehealth providers developed machine learning algorithms to assess the probability of a potential infection. If used adequately, these chatbots could play a major role in public health by slowing down the virus’s spread. With accurate answers, people could self-isolate early and contribute to better control of the pandemics.

In a recent study, Dennis et al. from Indiana University analyzed the user reactions to reputable COVID-19 screening chatbots. When chatbots are perceived to provide the same quality level as humans, users were more likely to use them and find them persuasive. 


Proactively informing the users of the potential bias and level of services they can expect is instrumental in ensuring the wide adoption of chatbots. 

To analyze if COVID-19 screening chatbots were reliable enough to be trusted and adopted by the users, the Austrian company Symptoma collaborated with the Paracelsus Medical University in Salzburg, Austria, recently published a study to evaluate and compare the diagnostic accuracies of COVID-19 specialized health chatbots. The results are quite striking.

[visualizer id=”643″ lazy=”no” class=””]


Ten freely available healthcare chatbots were selected and put to the test using 460 clinical cases for the correct evaluation of COVID-19. Three trained physicians extracted the symptoms for 50 confirmed Covid-19. 410 cases with at least one of the symptoms of COVID-19, as reported by the World Health Organisation (WHO), were used as the control group.

In case some of the questions asked by the symptom checker were not referenced, the “I don’t know” answer was chosen. The sensitivity and specificity of the diagnostic provided by the health chatbot were evaluated. In medical diagnosis, sensitivity refers to a test’s ability to correctly identify disease, whereas specificity is the test’s ability to correctly identify those without the disease.


Only 2/10 symptom checkers showed a reasonably good balance between sensitivity and specificity. Most other checkers were either too sensitive, classifying almost all patients as COVID-19–positive, or too specific, classifying many patients with COVID-19 as COVID-19–negative.

In the case of COVID-19, a web-based assessment cannot fully replace a polymerase chain reaction (PCR) test. Some people are asymptomatic, while others presenting with particular COVID-19 symptoms may, in fact, have a very similar but different disease.

Regardless, web-based COVID-19 symptom checkers are useful as a first triage buffer to avoid in-person physician visits or ease hospitals’ pressure. According to Munsch et al., “Symptom checkers could even replace telephone triage lines in which non–medically trained personnel read a predefined sequence of questions.” 

In 2019, from the University of Oxford, John Powell published the article “Trust Me, I’m a Chatbot: How Artificial Intelligence in Health Care fails the Turing Test” in the Journal of Medical Internet Research, stating than medicine requires wisdom more than intelligence. According to him, Artificial intelligence in healthcare needs to supplement rather than replace real doctors. We could not agree more.

AI-powered medical chatbots are useful tools to provide the first assessment but will never show empathy. As for triage, Chatbots can be reliable partners. Telemedicine companies could certainly leverage on them to assist doctors during virtual consultations with medical professionals.


Munsch N, Martin A, Gruarin S, Nateqi J, Abdarahmane I, Weingartner-Ortner R, Knapp B
Diagnostic Accuracy of Web-Based COVID-19 Symptom Checkers: Comparison Study
J Med Internet Res 2020;22(10):e21299
DOI: 10.2196/21299
PMID: 33001828

Was this article helpful?