Imaging Diagnosis vs. Gold Standard
PQI Project
DEFINE
Gap: Making accurate interpretations of imaging studies is of paramount importance to effective and efficient patient care. Consistently accurate imaging diagnoses are an essential element in clinical practice. Comparison of these imaging diagnoses with a gold standard offers the opportunity to learn from prior cases and improve practice performance. In this quality improvement project, radiology practices will compare their diagnostic interpretations to applicable gold standards of diagnosis. Examples of potential comparisons include, but are not limited to:
- CT diagnosis of appendicitis vs. operative results
- MRI diagnosis of meniscal tear vs. arthroscopy results
- Any other imaging finding vs. operative or pathologic finding
Target: The literature is replete with studies comparing imaging findings to a designated gold standard. Depending on the imaging finding and corresponding gold standard chosen, the diplomat will review the literature and choose the most similar study for which to establish their goal for performance. If there is no pertinent literature value, the diplomat will choose a goal that represents a 10% improvement from their baseline performance.
Timeline: This project is expected to take a minimum of 3 months to complete.
Stakeholder analysis and input: Obtaining stakeholder input from referring physicians and from radiology staff members is encouraged at the beginning of the project.
Potential impact: This project aligns with national goals of providing the best care to every patient every day. Inaccurate radiologic diagnoses can cause unneeded procedures and additional imaging for patients, can increase health expenditures, and can delay appropriate patient care.
MEASURE
Baseline: At baseline, <<your baseline rate entered here>> of the chosen imaging diagnosis is concordant with the gold standard. For example, a radiologist who diagnosed appendicitis 10 times (denominator) that was then operatively confirmed as being correct 9 times (numerator) and incorrect 1 time during the designated time period would report their performance as 90%. Any imaging diagnoses that did not have a gold standard performed for confirmation will be eliminated from the denominator.
Data Source: Each practice will determine their optimal data source. Ideally, an automated keyword search of the electronic medical record will be utilized to identify radiology reports containing the imaging diagnosis being tracked. The reports will be reviewed to assess for accurate documentation of the imaging diagnosis in question and then the medical record will be reviewed for the presence of a gold standard comparison. Initial screening of cases may be performed by an allied health staff assistant.
Sample Size: The baseline sample size is 25 or more radiology reports.
Counterbalance Measure: Since increased attention on a particular diagnosis may cause increased performance anxiety, the radiologist may spend more time interpreting these studies. Thus, an optional countermeasure would be that the report turnaround time (time from images available to report finalized) would not increase by more than 10%. Having radiology reports rapidly available to referring physician is a high satisfier for the referring physicians and is, of course, very important for timely patient care.
ANALYZE
Factors contributing to gap: The following are common contributors to the gap in performance. Each diplomate will identify the gaps present in their own practice.
• Radiologist was unaware of suspected clinical diagnosis, increasing chance of diagnosis being overlooked
• Suboptimal image quality due to technical or patient compliance issues
• Radiologist was pressed for time, increasing chance of diagnosis being overlooked
• Radiologist unaware of current imaging interpretation recommendations
• Human error or expected limitation of imaging
Rationale for choosing interventions: Review of the factors contributing to gap will be used to choose each intervention. Each radiology practice will have unique challenges and suspected reasons for decreased performance. Based on this analysis, each practice will chose interventions as detailed in the Improve section.
Quality improvement methodology used: The "5 Whys" technique from the Analyze phase of the Six Sigma DMAIC process will be utilized to assess reasons the primary interpretations were not accurate. At least two rapid, sequential Plan-Do-Study-Act (PDSA) cycles will be utilized to improve performance.
IMPROVE
Interventions implemented: The following are common interventions implemented. Each diplomate will customize interventions in their own practice.
• Education regarding accurate interpretion of imaging findings
• Review of prior misdiagnosed cases
• Change in imaging protocol
Comparison group: The comparison group is the individual's baseline performance. The diplomate will also be able to view their performance compared with a running national average of others completing this project via the MOCcredit.com web site.
Re-measurement results:
• After the first PDSA cycle, compliance was <<your first re-measurement rate entered here>> %
• After the second PDSA cycle, compliance was <<your second re-measurement rate entered here>> %
• Additional PDSA cycles can be optionally performed
Outcome measure: An optional additional measure of patient clinical outcome can be performed pre and post intervention.
CONTROL
Lessons learned: Common lessons learned from this project are as follows. Most radiologists have a tremendous amount of pride in their work. It can be difficult to ask people to change, since this is often taken as a criticism regarding their prior work. It is of utmost importance to approach changes such as these with a receptive customer-centered attitude. Our referring physicians and patients are our customers and if there is something we can do to improve our value, then we should work together to make it happen.
Barriers to implementation: The most common barrier to implementation is resistance to change. Essentially all radiologists have a preferred way to interpret studies that at least slightly differs from their colleagues. These "old habits" are hard to break and it is even harder if the radiologist does not recognize or accept that there is a more accurate way to perform their interpretations. Another barrier to implementation is suboptimal imaging. Modern imaging machines, dedicated imaging protocols, and skilled technologists to acquire the images are expensive and may not be available to practices where resources are scarce.
Communication Plan: The final project results will be anonymously shared nationally on MOCcredit.com.
Transition Plan: This project will be closed after successful completion with ownership to the person or group who maintains monitoring of imaging study interpretation accuracy.
Control Plan: The percentage of radiology reports that accurately diagnose the chosen entity is recommended to be sampled at least annually. If a decrease in performance to baseline levels is detected, an analysis of the data should be performed to generate a plan to return to the target performance.
Financial benefits: The financial benefits are indirect but include decreased expense for un-needed additional clinical and imaging workup in response to incorrect imaging diagnoses. Financial benefits also include decreased legal risk for misdiagnoses.
