Diagnostic Precision and Accuracy Using the TEED and O'Neill Clinical Grading Systems
Study Details
Study Description
Brief Summary
This validation study investigates the diagnostic precision and accuracy when grading otic barotrauma using two alternative gradings systems including the TEED and the OGS grading systems.
Background
There are two alternative grading systems used to grade otic barotrauma. They include the Teed Score (TEED) and the O'Neill Grading System (OGS). Teed has been utilized more extensively but has been found to have limitations among clinicians. The OGS is an alternative grading system with less classification categories than the Teed utilizes. The OGS system is tied directly to treatment decision making, specific to the respective grading level.
Methods
This study will utilize a sample of clinical examiners/observers who will independently grade 120 digital color photos of actual patient tympanic membranes with and without evidence of eustachian tube dysfunction and middle ear barotrauma including variable stages or grades of otic barotrauma. They will use the TEED chart as a reference in selecting the appropriate TEED for each of the photos. Simultaneously they will also utilize the OGS chart as a reference in selecting the appropriate OGS grade for each of the photos. The grading will be conducted by physicians, nurses, and technicians. This will allow for comparisons at varying provider levels. The comparisons will be made using the kappa statistics across the disciplines. This will allow comparison between observed agreement, and expected agreement due to chance for all reviewers.
Research Hypothesis
Ho: There is no difference in diagnostic precision and accuracy when grading otic barotrauma between examiners/observers using either the TEED or OGS grading systems.
Ha: There is a difference in diagnostic precision and accuracy when grading otic barotrauma between examiners/observers using either the TEED or OGS grading systems.
| Condition or Disease | Intervention/Treatment | Phase |
|---|---|---|
|
Detailed Description
Background
Eustachian tube dysfunction (ETD) and middle ear barotrauma (MEB) are the two most common complications of clinical hyperbaric oxygen treatment (HBO2T) though these disorders can present in many other settings. There are currently two alternative systems used to grade otic barotrauma in patients: the Teed Score (TEED) and the O'Neill Grading System (OGS). The TEED is the most commonly used grading system originally described in 1944 and contains five grades based on a single physician examination (1). There have been modifications to the TEED over the years included six grades, hence the TEED as modified by Macfie and Edmonds is commonly known as the Modified TEED Scale (2, 3). The TEED is not specific for the evaluation and treatment of middle ear barotrauma in patients undergoing HBO2T, pilots or airline passengers, soldiers exposed to explosive ordinance, recreational scuba divers, individuals driving to altitude, swimming injuries due to free diving to depths without equalization, loud sound pressure injuries, but rather to submariners practicing submarine escape training (1).
Otoscopy is commonly performed on all patients who present with these complaints and to hyperbaric oxygen patients prior to treatment. Though it is often again performed on patients presenting for follow-up and to hyperbaric patients after having completed treatment or those having difficulty equalizing middle ear pressure during the compression and/or decompression phase of the hyperbaric treatments. Although the results from these otoscopic examinations are used to determine the diagnosis and proper course of treatment for the ETD or MEB, the TEED may not correlate with the consistency of diagnosis, the clinical approach to relieving symptoms, or the definitive course of treatment for the otic barotrauma (2,3).
The OGS is a simplified classification system of ETD and MEB limiting the evaluation to three grade levels as opposed to six levels using the modified TEED (4).
Literature Review
A background search using PubMed without any restrictions on study type, date, language, country of origin, was carried out and no validation studies of either the TEED or the OGS grading system could be found. No studies comparing either of the two systems for consistency or uniformity of diagnoses were discovered or available for retrieval.
Specific Aims
We attempt to determine if the OGS and TEED can be used by all physicians including those seeing patients for suspected barotrauma in the community setting and the medical professionals who routinely see patients undergoing clinical hyperbaric oxygen treatments; hyperbaric physicians, nurses, and technologists, in order to determine if (1) both systems yield consistent results regardless of the system used for grading barotrauma and (2) will there be consistent results among the various professionals (physicians vs nurses vs. CHT's) performing the otoscopic evaluation. The goal of this study is to determine if using either the OGS or TEED will provide consistent and uniform results in grading barotrauma.
Study Question and Research Hypothesis
Is there a difference in precision and accuracy when scoring and grading ETD and MEB in patients between clinical staff members based on the type of scoring/grading system used?
Ho: There is no difference in precision or accuracy when grading ETD and MEB between examiners using either the TEED or the OGS grading systems.
Ha: There is a difference in precision and accuracy when grading ETD and MEB between examiners using either the TEED or the OGS grading system.
Ho: There is no difference in diagnostic precision and accuracy when grading ETD and MEB based on training level of examiners using either the TEED or the OGS grading system.
Ha: There is no difference in diagnostic precision and accuracy when grading ETD and MEB based on training level of examiners using either the TEED or the OGS grading system.
Methods
Inclusion criteria
All patients photographed between March of 2013 and June of 2019 that developed symptoms undergoing hyperbaric oxygen therapy were selected and deidentified. This involved retrieving all documented photographic images of ETD and MEB to score/grade the degree of otic barotrauma. These patients all had baseline photographs taken of the external ear canal and tympanic membrane as part of the pre-treatment standardized protocol in our department. Baseline photographs (normal) are included in the retrieved images. The photographs were initially screened and any images that were blurry to the degree that rendered scoring/grading impossible were excluded. Images considered interpretable despite blurring or partial obstruction due to cerumen or hair were included. This was done to maximize the use of photos that reflect the varying quality of images encountered in everyday clinical treatment conditions using otoscopy. Physicians were chosen in the pediatric and adult settings in a variety of fields including those in a hyperbaric clinical setting. The candidate physicians, hyperbaric nurses, nurse practitioners, and physician assistants were not limited to any specialty, however at the technician level the study was limited to those working in the hyperbaric medicine setting.
Sample size
The participants will review and independently grade a total of 120 otoscopic photos of tympanic membranes taken between March 2013 through June 2019. The photos include normal and abnormal images representing tympanic membranes with all levels of Eustachian tube dysfunction or middle ear barotrauma following hyperbaric pressurization that required a hold due to patient subjective ear symptoms.
The photo images were taken from actual hyperbaric patients seen at our facility with the exception of 10 photos. The 10 photographs included from outside sources were necessary to ensure inclusion of all stages of otic barotrauma including severe stages.
All participants in the study will be provided with instructions regarding the study both verbally and in writing. The study participants will have various levels of professional clinical training and will include physicians, nurses, and technologists.
Participants will use the modified TEED placard commonly used in hyperbaric centers as a reference in selecting the appropriate TEED score for each of the photos. Simultaneously, they will utilize the OGS placard as a reference in selecting the appropriate OGS grade for each of the photos. The grading will be conducted by hyperbaric physicians, nurses, and hyperbaric technicians (CHT's).
The TEED will be used without the benefit of a baseline photo, which was not required when this system was developed in 1944 and subsequently modified. The OGS system will be used initially without the use of a baseline photo, this is similar to the intended use of the TEED. The OGS system will subsequently be graded separately with the benefit of a comparison baseline photo in accordance with the intended use in hyperbaric treatment centers. Baseline photos could be obtained in other clinical settings beyond hyperbaric treatment settings.
Data Analysis
There will be three levels of comparison of precision and accuracy in using these two scales. The first comparison will evaluate the consistency displayed using each grading system (TEED and OGS) independently without the knowledge of the baseline status of the tympanic membrane prior to trauma. The second level of comparison will evaluate both grading systems for precision and accuracy as compared to each other. The third level of comparison will have the reviewers compare a baseline photo of the patient's tympanic membrane taken prior to hyperbaric treatment and pressurization to the post treatment traumatic photo and provide an OGS grade as the OGS was intended to be used in hyperbaric patients or in other settings where an available baseline photo may be available.
The analysis will include the use of a kappa statistic for comparison between observed agreement and expected agreement for each of the scoring/grading systems comparing examiners/observers reviewing the images (5). The Fleiss kappa statistic will be used for comparison since 3 or more examiners observers will score and grade the photo images (6).
A comparison of the kappa statistics with respect to scoring/grading system used, and level of training will be analyzed.
Ethical Considerations
The study uses de-identified archived photos of external ear canals and tympanic membranes. A protocol was submitted to the IRB and approved prior to the conduct of this study.
Funding Sources
None
Study Design
Outcome Measures
Primary Outcome Measures
- Teed Score [3 months]
Grade score by Healthcare provider, Teed Grade 0 - 5
- O'Neill Grading System (OGS) [3 months]
Grade score by Healthcare provider. O'Neill Grade 0 - 2
Eligibility Criteria
Criteria
Inclusion Criteria:
-
Physicians
-
Hyperbaric Nurses
-
Nurse Practitioners
-
HyperbaricTechnicians
Exclusion Criteria:
1-Refusal to participate upon invitation.
Contacts and Locations
Locations
| Site | City | State | Country | Postal Code | |
|---|---|---|---|---|---|
| 1 | Phelps Hospital | Sleepy Hollow | New York | United States | 10591 |
Sponsors and Collaborators
- Northwell Health
Investigators
- Principal Investigator: Owen J O'Neill, MD, MPH, Phelps Hospital Northwell Health
Study Documents (Full-Text)
None provided.More Information
Publications
- 1. Teed RW (Lieutenant Commander, USN). Factors Producing Obstruction of the Auditory Tube in Submarine Personnel. United States Navy Medical Bulletin, Washington: U.S. Govt. Print. Off. Vol. XLII, No. 2. February 1944, pgs. 293-306. Bureau of Medicine and Surgery, Navy Department, Washington, D.C.
- 2. Edmonds C, Bennett M, Lippman J, Mitchell SJ. Ear Barotrauma. Ch. 7, Pgs. 86 - 87. Diving and Subaquatic Medicine 5th ed.
- MACFIE DD. E.N.T PROBLEMS OF DIVING. Med Serv J Can. 1964 Dec;20:845-61.
- McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276-82.
- Nelson KP, Edwards D. Measures of agreement between many raters for ordinal classifications. Stat Med. 2015 Oct 15;34(23):3116-32. doi: 10.1002/sim.6546. Epub 2015 Jun 21.
- O'Neill OJ, Weitzner ED. The O'Neill grading system for evaluation of the tympanic membrane: A practical approach for clinical hyperbaric patients. Undersea Hyperb Med. 2015 May-Jun;42(3):265-71.
- 19-0868