The artificial intelligence model outperforms clinicians in diagnosing ear infections in children

An artificial intelligence (AI) model developed at Mass Eye and Ear was shown to be significantly more accurate in diagnosing pediatric ear infections than physicians reported in clinical use in the first head-to-head assessment of its kind, for which a research team worked to develop the model.

According to a new study published Aug. 16 in Otolaryngology – Head and Neck Surgery, the model, dubbed OtoDX, was more than 95 percent accurate in diagnosing an ear infection across a series of 22 test images, compared to an accuracy of 65 percent in a group of clinicians consisting of ENT doctors, paediatricians and general practitioners reviewing the same images.

When tested on a data set of more than 600 inner ear images, the AI ​​model had a diagnostic accuracy of more than 80 percent, a significant jump over the average accuracy reported by clinicians in the medical literature.

The model uses a type of AI called deep learning and was created from hundreds of photos collected from children prior to surgery at Mass Eye and Ear for recurring ear infections or fluid in their ears. The results represent a major step towards the development of a diagnostic tool that may one day be used in clinics to help physicians assess patients, the authors said. An AI-based diagnostic tool can offer providers such as pediatricians and emergency departments an additional test to better inform their clinical decision-making.

“Ear infections are incredibly common in children, but they are often misdiagnosed, leading to treatment delays or unnecessary antibiotic prescriptions,” said study lead author Matthew Crowson, MD, ENT and artificial intelligence researcher at Mass Eye and Ear and Assistant Professor of Otorhinolaryngology. Head and Neck Surgery at Harvard Medical School. “This model does not replace physicians’ judgment, but can serve to complement their expertise and give them more confidence in their treatment decisions.”

Common condition difficult to diagnose

Ear infections result from a buildup of bacteria in the middle ear. According to the National Institute on Deafness and Other Communication Disorders, at least five out of six children in the United States have had at least one ear infection before the age of three. Left untreated, ear infections can lead to hearing loss, developmental delays, complications such as meningitis, and in some developing countries, death. Conversely, over-treating children when they don’t have an ear infection can lead to antibiotic resistance, rendering the medication ineffective against future infections. This latter problem is of significant public health concern.

To ensure the best outcomes for children, doctors need to diagnose ear infections as accurately and early as possible. However, previous studies suggest that the conventional diagnostic accuracy of ear infections in children on a routine physical exam is below 70 percent, even with innovations in technology and clinical practice guidelines. The difficulty of evaluating a child who is struggling or crying during an exam, coupled with the general inexperience that many doctors and emergency medical professionals have with in-ear evaluations, according to Dr. Crowson explain the lower than expected diagnosis rate.

“Since clinicians prefer to err on the side of caution, it’s pretty easy to see why parents typically walk out of emergency care with a prescription for antibiotics,” he said.

In 2021, Dr. Crowson collaborated with Mass Eye and Ear colleagues Michael S. Cohen, MD, director of the Multidisciplinary Clinic for Pediatric Hearing Loss, and Christopher J. Hartnick, MD, MS, director of the Division of Pediatric Ear, Nose and Throat to provide a more specific Method for diagnosing ear infections using a machine learning algorithm. An artificial neural network was trained using high-resolution photographs of eardrums collected directly from patients undergoing ear surgery where infection can be seen. These photos represent a gold standard, “ground truth” dataset when compared to AI-based tools that rely on images collected from search engines. A proof-of-concept study published last year found that the model is 84 percent accurate in detecting “normal” versus “abnormal” middle ears.

human versus machine

In the new study, researchers directly compared the accuracy of a refined model to clinicians. More than 639 images of eardrums from children 18 years of age and younger undergoing surgery to place tubes or drain fluid from the ears were used to train the model. The images were labeled as either “normal,” “infected,” or with “fluid behind the eardrum,” as opposed to the “normal” or “abnormal” classification of the team’s earlier model. With the added segment, the model achieved a mean diagnostic accuracy of 80.8 percent.

A survey was then created in which physicians and trainees from various medical specialties were asked to view 22 new images of eardrums and to diagnose the ear as one of the three categories marked. While the machine learning model correctly categorized more than 95 percent of the sample images, the average diagnostic score was 65 percent among 39 physicians who took part in the survey. In addition, pediatricians and general practitioners/general internists correctly categorized 60.1 percent and 59.1 percent of the images, respectively.

Bringing artificial intelligence to the clinic

Ongoing studies are underway to validate and refine the AI ​​model. To date, more than 1,000 intraoperative images of the eardrum have been collected at Mass Eye and Ear.

In partnership with Mass General Brigham Innovation, OtoDx is currently being deployed in a prototype device paired with a smartphone app. The device acts as a “mini otoscope” that fits over the phone’s camera and allows doctors to take photos of the inside of a child’s ear, upload them straight to the app and get a diagnostic reading in seconds. With further validation, OtoDX can provide another tool for physicians to get real-time information during an exam.

As feedback is processed for the pilot, Mass General Brigham Innovation will support the OtoDx team in exploring ways to commercialize this powerful tool to help even more physicians and their patients.

In addition to Dr. Crowson, Cohen, and Hartnick included study co-authors Krish Suresh, MD, of Mass Eye and Ear/Harvard Medical School, and David W. Bates, MD, MSc, of Brigham and Women’s Hospital/Harvard TH Chan School of public health.

This study was supported in part by a grant from the National Institutes of Health Biomedical Informatics and Data Science Training Program. (T15LM007092-30). The technology is the subject of a pending patent application.

About mass eye and ear

Massachusetts Eye and Ear, founded in 1824, is an international treatment and research center and teaching hospital of Harvard Medical School. A member of Mass General Brigham