DOI: 10.1148/rg.281075051
RadioGraphics 2008;28:309-316
© RSNA, 2008
Informatics in Radiology
Computer-based Simulator for Radiology: An Educational Tool
Alexander J. Towbin, MD,
Brian E. Paterson, BA, and
Paul J. Chang, MD
1 From the Department of Radiology, University of Pittsburgh Medical Center, Pittsburgh, Pa. Presented as an Informatics exhibit at the 2006 RSNA Annual Meeting. Received March 23, 2007; revision requested June 13 and received August 3; accepted August 16. P.J.C. is a cofounder of Stentor (subsequently purchased by Koninklijke Philips Electronics NV); the remaining authors have no financial relationships to disclose.
Address correspondence to A.J.T., Department of Radiology, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, Cincinnati, OH 45229 (e-mail: towbinaj{at}gmail.com).
 |
Abstract
|
|---|
In the past decade, radiology has moved from being predominantly film based to predominantly digital. Although in clinical terms the transition has been relatively smooth, the method in which radiology is taught has not kept pace. Simulator programs have proved effective in other specialties as a method for teaching a specific skill set. Because many radiologists already work in the digital environment, a simulator could easily and safely be integrated with a picture archiving and communication system (PACS) and become a powerful tool for radiology education. Thus, a simulator program was designed for the specific purpose of giving residents practice in reading images independently, thereby helping them to prepare more fully for the rigors of being on call. The program is similar to a typical PACS, thus allowing a more interactive learning process, and closely mimics the real-world practice of radiology to help prepare the user for a variety of clinical scenarios. Besides education, other possible uses include certification, testing, and the creation of teaching files.
© RSNA, 2008
|
Introduction
|
|---|
In the current health care environment, there has been an increasing push toward the competency-based practice of medicine, in which physicians must show that they have been adequately trained or prepared to provide a specific level of care before being permitted to provide that level of care. Although this movement has been directed toward better patient care, the question arises as to how one can gain experience without already having had experience.
This trend has been most evident in residents’ preparation to take call. Current resident preparation includes "core" rotations, dedicated lectures, "mini-call," and review of teaching files. Although each method is useful in imparting knowledge, none mimics the resident’s on-call task: independent interpretation of a complete series of images. To address this deficiency, we created a simulator for radiology.
Simulators have been used in both aviation and medicine to train users for specific scenarios. In medicine, simulators have been used mostly to help teach a specific technique or skill such as anesthetic administration, laparoscopic surgery, and colonoscopy (1–4). In radiology, simulators have been used to teach interventional and sonographic techniques as well as optimal film acquisition (5–9). However, few simulators have been described that mimic the day-to-day practice of radiology—namely, image interpretation (10–13)—and each of these simulators has been used on a limited scale.
The ideal radiology simulator should have several key components. First, it should closely approximate the appearance and functionality of the picture archiving and communication system (PACS) in use at the hospital. This similarity allows the user to become familiar with the specific features of the PACS, including image manipulation, measurement, and annotation. With increased familiarity, the user is able to use the PACS tools quickly and efficiently. With the simulator mimicking the hospital PACS, the user is able to train in an environment that is similar to everyday practice.
Second, the ideal simulator should accept fully anonymized Digital Imaging and Communications in Medicine (DICOM)–compliant studies. Having the full DICOM data set allows users to manipulate the images as they would in clinical practice. The ability to perform tasks such as changing the window width and level, taking measurements, and recording attenuation levels in Hounsfield units allows the radiologist to make a more specific diagnosis. The anonymization of cases ensures compliance with the Health Insurance Portability and Accountability Act.
One of the most important elements of an ideal simulator is that it should contain the entire study as acquired. By including every image of each sequence, the simulator is again able to mimic the everyday practice of radiology. Each image and sequence can be used to look for pertinent positive and negative findings. Thus, users are able to narrow the differential diagnosis down to a more specific diagnosis.
Fourth, in an ideal simulator, case entry and retrieval should be simple. A current limitation of many teaching systems is that case entry is clumsy and time consuming, thereby limiting the incentive to add new cases. In the digital era, many of the data already exist and can easily be acquired and transferred to a simulator program. The ideal simulator would allow fully automated case entry directly from the PACS. Along with easy case entry, the ideal simulator must allow straightforward selection of task-specific or level-appropriate cases. The ability to select a specific type of case allows users to focus their studies.
Finally, the ideal simulator should provide both immediate and long-term feedback to augment the education process. Immediate feedback would include showing users the "answer" to the case as dictated by a specialist radiologist. Long-term feedback allows users to discover their areas of weakness so that they can concentrate their further studies. Both of these key components were considered and implemented in the construction of our simulator.
In this article, we discuss and illustrate the components of the ideal simulator as well as the design, implementation, and possible applications of our simulator program.
|
Case Entry
|
|---|
A case manager was created to facilitate case entry. This program is Web based and functions within the local hospital intranet with the use of Internet Explorer (Microsoft, Redmond, Wash). The entered data are stored in a local database with the use of SQL (Microsoft).
After a case is selected for the simulator, it is sent to an honest broker (ie, a disinterested third party) to be anonymized and scrubbed of any patient-identifying data. If comparison studies are desired, they would have to be selected, anonymized, and added separately. At our hospital, the anonymized cases are placed on a dedicated research server so that they cannot be linked back to a specific patient. Thus, although the anonymized version of a case is not present on the clinical server, the original patient data remain on the server and clinical work flow is not affected.
Once a case has been anonymized, the anonymized medical record number and accession number are entered into the case manager. Additional information is entered and includes pertinent history; modality; division (subspecialty within radiology, such as neuroradiology or abdominal radiology); disease process (eg, infection, trauma, neoplasm); call level (junior or senior); and difficulty (easy, medium, or hard) (Fig 1a).
View larger version (43K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1a. Case entry. (a) First, the anonymized medical record number and accession number are entered, followed by the pertinent patient history. The case is categorized according to modality, division, category (disease process), call level, and difficulty with the use of drop-down menus. If a new modality, division, or category is required, it can be created at the bottom of the page. (b) Once the case has been created, the relevant findings are added in order of importance.
|
|
View larger version (36K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1b. Case entry. (a) First, the anonymized medical record number and accession number are entered, followed by the pertinent patient history. The case is categorized according to modality, division, category (disease process), call level, and difficulty with the use of drop-down menus. If a new modality, division, or category is required, it can be created at the bottom of the page. (b) Once the case has been created, the relevant findings are added in order of importance.
|
|
The data are entered at this step with use of drop-down menus that are already populated with specific data points. If a new subheading is needed, it can be added instantly. These data are then stored in the database. The specific categorization of each case has two advantages: (a) it allows the user to search for more specific cases, and (b) it enables the program to provide more specific feedback and statistics.
Once a case has been entered and categorized, the case findings can be entered (Fig 1b). The findings are cut and pasted from the impressions section of the final report, which has been dictated and signed by a specialist radiologist. Multiple findings can be entered depending on the complexity of the case. The findings are entered in order of importance, after which the case is ready to be viewed in the simulator program.
Although the case entry process is not automated, it is rapid. Once the cases have been anonymized, approximately 25 cases per hour can be entered by a single user. Further refinements to the program could improve the speed of entry. Currently, the bottleneck in the process is the anonymization of cases. Some of the categories (eg, pertinent history, modality, division) can be filled in from the DICOM data set, whereas others (eg, call level, disease process, difficulty) must be user defined. Ideal case entry would prepopulate each category but allow the user to modify the category as he or she sees fit.
Although entry of findings requires only a little work, this process could be automated as well. If each impression from the dictated report were automatically entered into the findings section, the only remaining task for the user would be to edit and order the findings.
|
Simulator Use
|
|---|
The simulator was created to be integrated with the Stentor PACS (Philips, Brisbane, Calif), which is used at our hospital. Because the simulator has a recognizable PACS as a "backbone," only the interface had to be developed. With use of hypertext markup language, the simulator was designed as a "shell" program to work in concert with the PACS.
Although no patient-specific data are retained in the simulator, the user must first log in with a unique user name and password. After log-in, the main work page appears. The major difference between the simulator and the PACS is the appearance of this case work list. The key feature added to the work list on the simulator is the ability to search for studies on the basis of modality, division, disease, call level, and difficulty (Fig 2). Although these categories are designed specifically for our purpose (ie, call preparation), they could easily be modified to fit any purpose.
After the user selects specific types of cases for viewing, the case list is populated with relevant cases (Fig 3). If the user prefers to view cases randomly, each drop-down menu can be set to "All" and all of the cases displayed. The user then selects a case by clicking on the case description. The entire study is immediately opened in a familiar PACS interface (Fig 4a). At this point, the simulator again diverges slightly from the typical PACS: A pop-up window appears that contains the pertinent history and a text box in which the user can enter his or her impressions (Fig 4b).
View larger version (38K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 4a. Case review and response grading. (a) Once a case has been selected, the entire study is opened in a familiar PACS interface. (b) Unlike with the typical PACS, a pop-up window appears that contains the pertinent history and a text box in which to enter findings. (c) Once the user has reviewed the images, the findings can be entered in the text box. (d) After submitting an interpretation, the user is presented with the impression from the actual dictated interpretation. The user must then grade his or her own response for accuracy and check to see whether his or her primary impression matches the primary dictated impression. (Fig 4 used with permission from Philips.)
|
|
View larger version (15K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 4b. Case review and response grading. (a) Once a case has been selected, the entire study is opened in a familiar PACS interface. (b) Unlike with the typical PACS, a pop-up window appears that contains the pertinent history and a text box in which to enter findings. (c) Once the user has reviewed the images, the findings can be entered in the text box. (d) After submitting an interpretation, the user is presented with the impression from the actual dictated interpretation. The user must then grade his or her own response for accuracy and check to see whether his or her primary impression matches the primary dictated impression. (Fig 4 used with permission from Philips.)
|
|
View larger version (19K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 4c. Case review and response grading. (a) Once a case has been selected, the entire study is opened in a familiar PACS interface. (b) Unlike with the typical PACS, a pop-up window appears that contains the pertinent history and a text box in which to enter findings. (c) Once the user has reviewed the images, the findings can be entered in the text box. (d) After submitting an interpretation, the user is presented with the impression from the actual dictated interpretation. The user must then grade his or her own response for accuracy and check to see whether his or her primary impression matches the primary dictated impression. (Fig 4 used with permission from Philips.)
|
|
View larger version (29K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 4d. Case review and response grading. (a) Once a case has been selected, the entire study is opened in a familiar PACS interface. (b) Unlike with the typical PACS, a pop-up window appears that contains the pertinent history and a text box in which to enter findings. (c) Once the user has reviewed the images, the findings can be entered in the text box. (d) After submitting an interpretation, the user is presented with the impression from the actual dictated interpretation. The user must then grade his or her own response for accuracy and check to see whether his or her primary impression matches the primary dictated impression. (Fig 4 used with permission from Philips.)
|
|
The user then reviews the case. All of the functionality that is present in a typical PACS is present in the simulator. The user can manipulate the images as needed to facilitate the diagnostic process. Once the user has reviewed the images and made the findings, an interpretation can be entered in the text box (Fig 4c). The simulator accepts free-text entry. If multiple findings are present, the user clicks the "Next Finding" button. Once there are no more findings to enter, the user clicks the "Submit Findings" button.
The user is then able to get immediate feedback. The pop-up window changes from one in which the user can enter findings to the answer window. The user’s interpretation is listed on one side, and the interpretation from the actual dictated report, approved by the specialist radiologist, is listed on the other (Fig 4d). The user then grades his or her impressions for accuracy. In addition, because of the importance of stating the most important finding first when on call, the user must check a box if his or her first impression is also listed first as the primary impression. During the grading process, the user has the opportunity to review the images to determine why a finding was missed or how the findings fit together.
Free-text entry was used for entering the findings because it allows the most flexibility, requires the most thought, and most closely mimics the on-call process already in place. The other options—namely, multiple choice, selection of a diagnosis from a massive list, or structured terminology such as SNOMED (Systematized Nomenclature of Medicine) or RadLex (Radiology Lexicon)—were thought to be cumbersome and unrealistic compared with the daily work flow. Furthermore, a multiple-choice question limits the potential diagnosis to the number of possible answers, unrealistically focusing the search on specific entities and thus defeating the main purpose of the simulator.
Free-text entry made the grading process more difficult, since the ideal simulator would allow automatic grading of each case. Automatic grading of free text requires a significant amount of additional programming with only a limited improvement in functionality. To correctly grade a free-text response, every possible abbreviation, misspelling, and word choice would have to be accounted for; otherwise, a false, incorrect grade may be assigned. In the end, self-grading was thought to be the simplest method. The user has incentive to grade each case accurately because he or she can use the statistics generated in the performance report to find areas of weakness. There is also little incentive to cheat, since no one else has access to the user’s results. If the simulator were being used for testing purposes, grading would have to be performed by a third party. Such third-party grading is indeed possible, since each response entered by the user is saved to the database.
After finishing the case, the user clicks the "Submit" button and is returned to the case list. Completed cases are grayed out; however, a case can be viewed multiple times.
At any time during the use of the simulator, the user can obtain a performance report (Fig 5). This report is generated using data collected from simulator use. For each case, specific data are collected, including the findings entered, the user’s interpretation of the case, and the time spent on the case from start to finish. The performance report shows the user the total number of cases he or she viewed, the total percentage of correct findings, the total percentage of primary findings that were listed first, and the average time in seconds it took to complete each case. In addition, the data are broken down into more specific categories. The same data points are provided for each searchable category, including modality, division, and disease process. The user sees his or her own results as well as peer group results and is thus able to compare personal progress with that of the group, determine areas of weakness, and hone further studies.
View larger version (18K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 5. Performance report. The performance report is customized for the user’s specific experience. The report provides the user with data regarding his or her accuracy, the number of cases viewed, and the average time spent interpreting each case. Overall data are provided and then broken down into subcategories. The user’s data are compared with those of his or her peer group as a whole, allowing the user to determine areas of weakness and focus further studies.
|
|
|
Discussion
|
|---|
There are many benefits associated with simulator use, both in general and specifically within radiology. Perhaps the most important benefit is safety. Simulators have been used in the aviation industry for nearly 80 years to help train pilots. By using simulators, pilots can practice responding to emergency situations so that, should such a situation arise, their responses are automatic (3). The corollary with medicine is obvious. In addition, simulators allow both pilots and physicians to practice without harming others, whether they are passengers or patients.
Simulators also allow education to be relatively standardized. Every user can simulate the same scenario or case; thus, all users have the same experience. This uniformity also allows a user to become familiar with rare situations. For example, pericardial tamponade is a potentially life-threatening condition that is occasionally seen at chest computed tomography (CT). A typical first-year resident may not come across a case of pericardial tamponade in his or her regular rotations or in conferences. Thus, if a case arises when the resident is acting independently on call, there is a high likelihood that he or she will miss the findings and incorrectly or incompletely interpret the images. However, because the case is presented on the simulator, the resident has already seen this uncommon condition and will be more likely to make the correct diagnosis.
A third benefit of simulators is that they make use of an active learning process. Active learning methods (eg, group discussion, practice by doing, teaching others) have higher rates of information retention compared with passive learning techniques (lecture, reading, or observation) (14). By making use of the active learning process, simulators help the user learn and retain information, which, along with increased practice, better prepares him or her to handle difficult cases when they arise.
There are many uses for simulator training within radiology besides call preparation and resident education. For example, a simulator can be used to teach seasoned radiologists a new modality. Few radiologists have experience with new modalities such as cardiac CT, cardiac magnetic resonance imaging, and nuclear medicine–radiology fusion examinations such as positron emission tomography–CT (PET/CT) or single photon emission CT–CT (SPECT/CT). With a simulator, a large number of radiologists can view a large volume of cases in a short time. After residency, it is difficult for a radiologist to gain experience with a new modality. Simulators could be used to help certify radiologists trying to gain this experience.
Another potential use of the simulator is for obtaining continuing medical education credits. For example, after reading an article on liver tumors, users could review 10 cases of patients with such tumors. They could then use their newfound knowledge to interpret the cases and solidify their understanding of each diagnosis.
Simulators can also be used as a testing tool. Applications might include maintenance of certification, testing competency in or mastery of a specific subject, or pretesting-posttesting in a specific residency rotation. One possible scenario is that residents would have to show competency before being allowed to take call. Competency could be proved by scoring above a certain level on a posttest administered after each core rotation. Then, before taking call, the resident would have to pass an overall test that has been created to simulate a typical call night. Each residency program could add cases specific to its institution and call responsibility. Although the validity of such a test would have to verified, the test would be a step toward competency-based medicine.
One significant potential use of the simulator is to augment or replace the current digital teaching file standard. One major drawback of the Medical Imaging Resource Center (MIRC) standard devised by the Radiological Society of North America, as well as of other teaching file systems, is that only a few selected images are added to each file. Although this approach is useful for quick review or identification of a specific finding, it is often suboptimal for several reasons: (a) multiple contiguous images are often required for clear depiction of the finding; (b) it is much easier to make a finding on a single image when the user knows an abnormality is present; and (c) one of the major purposes of modern radiology (ie, to make the finding on a series of images) is defeated. Other disadvantages of some programs using the MIRC standard include time-intensive case entry, difficult and nonstandard formatting, and the inability to manipulate images. The major advantages of the MIRC standard are that (a) there is an in-depth discussion of the case, and (b) the findings can be pointed out to the user. By combining the advantages of the simulator and the MIRC standard, an ideal teaching file could be created.
The major drawback of the current version of the simulator is that it is both vendor (Stentor-Philips) and institution (University of Pittsburgh Medical Center) dependent. An ideal version of the simulator would integrate easily into any PACS system and would be vendor independent. The program described in this article has been used in our radiology department for approximately 18 months to help residents prepare for being on call (15). In future versions of this program, we will try to address the vendor and institution dependence issues.
|
Conclusions
|
|---|
In the past 10 years, radiology has moved from being predominantly film based to predominantly digital. Although clinically the transition has been relatively seamless, the method in which radiology is taught has not kept pace. Simulators have the potential to advance radiology education with the use of an active learning process. Case-based simulators closely mimic the real-world practice of radiology and can help prepare the user for many specific scenarios. Within radiology, simulators have many possible uses beyond resident education, including certification, testing, and the creation of teaching files. Because many radiologists already work in the digital environment, a simulator could easily and safely be integrated with the PACS and become a powerful tool for radiology education.
|
TAKE-HOME POINTS
|
|---|
There are many benefits associated with simulator use, both in general and specifically within radiology. Perhaps the most important benefit is safety.
- Simulators also allow education to be relatively standardized.
- A third benefit of simulators is that they make use of an active learning process.
- There are many uses for simulator training within radiology besides call preparation and resident education.
- One significant potential use of the simulator is to augment or replace the current digital teaching file standard.
|
Footnotes
|
|---|
Abbreviations: DICOM = Digital Imaging and Communications in Medicine, MIRC = Medical Imaging Resource Center, PACS = picture archiving and communication system
|
References
|
|---|
- Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D. Virtual reality colonoscopy simulation: a compulsory practice for the future colonoscopist? Endoscopy 2005;37:1198–1204.[CrossRef][Medline]
- Schaefer JJ 3rd. Simulators and difficult airway management skills. Paediatr Anaesth 2004;14:28–37.[CrossRef][Medline]
- Wong AK. Full scale computer simulators in anesthesia training and evaluation. Can J Anaesth 2004;51:455–464.[Medline]
- Kolkman W, Van de Put MA, Van den Hout WB, Trimbos JB, Jansen FW. Implementation of the laparoscopic simulator in a gynecological residency curriculum. Surg Endosc 2007;21:1363–1368.[CrossRef][Medline]
- Monsky WL, Levine D, Mehta TS, et al. Using a sonographic simulator to assess residents before overnight call. AJR Am J Roentgenol 2002;178: 35–39.[Abstract/Free Full Text]
- Chaer RA, Derubertis BG, Lin SC, et al. Simulation improves resident performance in catheter-based intervention: results of a randomized, controlled study. Ann Surg 2006;244:343–352.[Medline]
- Berry M, Lystig T, Reznick R, Lonn L. Assessment of a virtual interventional simulator trainer. J Endovasc Ther 2006;13:237–243.[CrossRef][Medline]
- Fanti V, Marzeddu R, Massazza G, Randaccio P, Brunetti A, Golosio B. A simulator for X-ray images. Radiat Prot Dosimetry 2005;114:350–354.[Abstract/Free Full Text]
- Winslow M, Xu XG, Yazici B. Development of a simulator for radiographic image optimization. Comput Methods Programs Biomed 2005;78: 179–190.[CrossRef][Medline]
- Ganguli S, Pedrosa I, Yam CS, Appignani B, Siewert B, Kressel HY. I. Preparing first-year radiology residents and assessing their readiness for on-call responsibilities. Acad Radiol 2006;13: 764–769.[CrossRef][Medline]
- Yam CS, Kruskal J, Pedrosa I, Kressel H. II. Preparing and assessing first-year radiology resident on-call readiness technical implementation. Acad Radiol 2006;13:770–773.[CrossRef][Medline]
- LoRusso AP, Bassignani MJ, Harvey JA. Enhanced teaching of screening mammography using an electronic format. Acad Radiol 2006;13:782–788.[CrossRef][Medline]
- Toms AP, Kasmai B, Williams S, Wilson P. Building an anonymized catalogued radiology museum in PACS: a feasibility study. Br J Radiol 2006;79: 666–671.[Abstract/Free Full Text]
- Erinjeri JP, Bhalla S. Redefining radiology education for first-year medical students: shifting from a passive to an active case-based approach. Acad Radiol 2006;13:789–796.[CrossRef][Medline]
- Towbin AJ, Paterson BE, Chang PJ. A computer-based radiology simulator as a learning tool to help prepare first-year residents for being on call. Acad Radiol 2007;14:1271–1283.[CrossRef][Medline]
This article has been cited by other articles:
|
|
|
|
 
S. Ganguli, M. Camacho, C.-S. Yam, and I. Pedrosa
Preparing First-Year Radiology Residents and Assessing Their Readiness for On-Call Responsibilities: Results Over 5 Years
Am. J. Roentgenol.,
February 1, 2009;
192(2):
539 - 544.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Introduction
