(Radiographics. 2001;21:1025-1032.)
© RSNA, 2001
Do Computers Teach Better? A Media Comparison Study for Case-based Teaching in Radiology1
Martin Maleck, MD, 2,
Martin R. Fischer, MD, 2,
Birgit Kammer, MD,
Claudius Zeiler, MD,
Eugen Mangel, MD,
Franz Schenk, MD and
Klaus-Juergen Pfeifer, MD
1 From the Departments of Medicine (AG Instruct) (M.M., M.R.F.), Diagnostic Radiology (M.M., B.K., E.M., F.S., K.J.P.), and Surgery (C.Z.), Klinikum Innenstadt, Ludwig Maximilians University, Ziemssenstrasse 1, 80336 Munich, Germany. Received April 21, 2000; revision requested June 20; final revision received October 19; accepted December 1. Address correspondence to M.M. (e-mail: martin.fischer@medinn.med.uni-muenchen.de).
 |
Abstract
|
|---|
A prospective study was performed to better define the role of computers in teaching radiology to medical students. Two hundred twenty-five 3rd-year students were randomly assigned to one of four groups and exposed to 10 radiology cases as well as to a voluntary weekly radiology lecture. Group A used computer-based cases with interactive elements; group B used computer-based cases without interactive elements; group C used paper-based cases with interactive elements; and group D was not exposed to the cases and served as a control group. On a multiple-choice question test, groups A, B, and C showed significant improvement (+11.2%, +15.1%, and +13.0%, respectively), whereas group D did not (+0.6%). On an image interpretation test, group A showed the most improvement (+15.7% [P < .001]), followed by group B (+15.1% [P < .01]) and group C (+10.2% [P < .05]); group D showed no significant improvement (+8.5%). No significant differences in the learning outcome were found between the two interactive groups (computer based and paper based). Computer-based teaching with case studies (with or without interactivity) improves students’ problem-solving ability in radiology.
Index Terms: Computers, educational aid • Education
|
Introduction
|
|---|
Research on computer-assisted instruction tools started at the beginning of the 1970s (1), but the practicality of their use in radiology is still unclear. There seems to be a discrepancy between the rich availability of software and computer technologies and the poor integration of these new tools into clinical teaching. Early reports on computer-assisted instruction in radiology were focused mainly on describing a new learning program or tool (2,3) or related to improvements in computer technology (4–6).
Early on, only a few studies explored the possibilities of embedding the computer into medical curricula (7,8). Computer-assisted instruction was evaluated by groups of users, mostly medical students, for acceptance and motivational aspects (9,10). It is now evident that this method of subjective self-evaluation would provoke much criticism from experts in education; however, it identified a gap in educational research (11–13).
Within the general discussion about "the way in which we teach" and the movement toward problem-based learning (14), the discussion about the best way to use computers in medical education became more and more important and led to studies that compared computer use with use of textbooks or traditional lectures (15–19). However, previous media comparison studies were limited by confounding factors (20). Some articles explained how to develop computer-assisted instruction tools yet ignored didactic developments (21,22).
With the rapid growth of the Internet community, computer-assisted instruction holds great promise for providing high-quality teaching materials that are readily available anywhere at any time. Distribution and updating of educational data should be easier, faster, and more economical. But can all students handle the technology adequately? Is the learning outcome at least as good as with the well-established traditional teaching approaches?
We performed a study that compared use of different versions of a computer-assisted instruction program with use of a paper-based version of the same teaching material. Factors for successful integration of computer-assisted instruction into clinical teaching in radiology should be identified and described.
|
Materials and Methods
|
|---|
We created 10 cases for the computer with the CASUS authoring system (23) (Fig 1) and a paper-based traditional version with copies of the original radiographs; the cases were created as an extension to the radiology lecture in the second clinical year at Ludwig Maximilians University. The cases were developed with contributions from experts in the fields of radiology, surgery, and internal medicine. Five cases focus on typical problems seen on chest radiographs (adult respiratory distress syndrome, lung carcinoma, tuberculosis, atelectasis, and pneumothorax). The other five cases cover problems seen at bone imaging (benign tumor, malignant tumor, fracture, bone infection, and degenerative bone disease). The structure of each case includes two screens (pages) of medical history and physical examination results. In the second part, four or five screens (pages) deal with the related imaging studies and a final screen refers to therapy and follow-up. In addition, hypertext links, graphics, audio, and a mapping tool to illustrate the differential diagnostic process are included. The time users spent on each case varied from 20 to 30 minutes.
All 225 students from the 1998 summer term participated in the study. The case studies were a mandatory part of the curriculum; the students were required to participate as a prerequisite for the surgery course in the next term. Selection bias could thus be excluded. The students had the options of attending radiology lectures (lecture duration, 45 min/wk) on a voluntary basis and of using textbooks. Students were randomly assigned to one of four instructional groups by using random selection based on radioactive decay numbers. The teaching content was identical for all groups.
Group A used the computer-based cases along with interactive elements, which included multiple-choice and free-text questions and a drag-and-drop mapping tool for the differential diagnostic process. Group B used the computer-based cases but without the interactive elements. Group C used the paper versions of the cases together with the original film radiographs. Group D was not exposed to the cases and served as a control group. (The students from group D were offered the opportunity to learn with the cases after the end of the term.)
The control group was important to avoid the negative effects of confounding factors. The exposition on learning cases deals only with groups A–C. Owing to organizational problems (overlapping courses), 20 students could not be included in group A, B, or C and had to be assigned to group D. A subgroup analysis showed no statistical differences between those 20 students and the students in the four study groups.
The 10 cases for the intervention groups were offered in two blocks of five cases. Each student had to work through five cases on Tuesday and five cases on Wednesday of the same week (Fig 2). We changed the case order and rotated the groups during the term to minimize confounding effects related to case order, time between case studying, and the posttest. The same tutor gave an introduction to all students on each course day. The students worked on the cases in groups of two or three.
All students took a pretest at the beginning of the semester and a posttest at the end of the semester; both the pretest and the posttest contained 14 multiple-choice questions originally used for part II of the National Boards Examination ("2. Staatsexamen") and four free-text questions related to interpretation of radiographs, which were projected with a slide projector. An answering time of 90 seconds was allowed for each multiple-choice question; an answering time of 2 minutes was allowed for each free-text question. The radiographs used in the tests were not the same as those used in the cases. To determine success-pressure effects, the students were told that they must take part only in the posttest and that they would not fail. We normalized the results of the multiple-choice questions and of the free-text questions to a 100% standard to exclude variations between the pretest and the posttest. The standard of reference for the multiple-choice questions was the score of the national examination. For the free-text questions, it was the interpretation of a radiology professor (K.J.P.). Before taking each test, the students had to fill out an evaluation form with about 35 items so that we could collect individual and subjective data related to motivational aspects.
For groups A and B, we used our Computer Learning Center, which has nine workstations(Macintosh Power PC 8200/120 [Apple Computer, Cupertino, Calif]). The computers were placed in a row and were divided by opaque walls, and all had multiple headphones. The computer cases were saved on a server and were used by the students in client-server mode. The paper-based group was placed in the appointment rooms of the outpatient clinic next door to the Computer Learning Center. Each group of two or three students had its own room with a radiographic viewing device. For all groups, the same student tutor was present at all times and gave the same standardized introduction before starting the course. No commentaries or help related to the cases was given outside of technical advice on use of the computers or software.
We used the F test for same variances and contingency tables to analyze the evaluation data and the Wilcoxon signed rank test for nonparametric data to evaluate the pre- and posttest results (categorical data).
|
Results
|
|---|
Nineteen students were physically not present for the whole semester for individual reasons like failed examinations, studies abroad, or illness. Thus, the total number of participants was 206 (92% of all students). Only students who completed the pretest and posttest (groups A–D) and at least one of the two course days (groups A–C) were included in the evaluation. The reasons why students did not complete the posttest are not known. The students who completed only half of the cases had to select atypical courses for individual reasons. Therefore, the number of students included in the study was 192 (85%). Group comparison (
2 table test) showed no significant differences between the groups for age, sex, priorprofession, or prior knowledge of problem-based learning (Table 1), nor differences in computer skills. This information was requested on the questionnaire the students had to fill out before they took the pretest.
Subjective Student Self-Evaluation
Self-evaluation forms filled out directly after each learning session showed a significant difference in the level of concentration while working on the cases between group B and groups A and C (P < .05 [contingency table analyses, which were used for all self-evaluation data]) (Table 2). The students in group C rated their concentration as highest, followed by groups A and B. In addition, group B rated the difficulty of the cases significantly lowest in comparison with groups A and C (P < .05). The adequacy of the available time for working on the cases was judged very differently by the three groups: Group B thought that there was enough time, whereas groups A and C mentioned time constraints as a major problem (P < .0001). Groups A and C rated the benefits from interactivity as high (89%), whereas 42% of the students in group B missed interactivity. The degree of "fun" while working on the cases was rated highest in group C, followed by groups A and B with significant differences (group A vs group C, P < .05; group B vs group C, P < .001).
At the end of the semester, use of a radiology textbook was rated low (mean of 1.7 on a scale from 1 to 6) by all of the groups, as was lecture attendance (1.9). However, lectures, textbooks, and the case-based radiology course subjectively played a different, supportive role for the improvement of knowledge in the different groups (Table 3). There was no significant difference between the groups concerning subjective improvement of knowledge. There was a large variance in all of the groups in evaluating the role of the textbook. We asked the students whether they would recommend the course to peers. Group A answered this question positively (mean of 4.5 on a scale from 1 to 6), as did group B (4.4) and group C (5.0). There was a significant difference only between groups B and C (P < .05).
Objective Outcome Measures
The scores for the multiple-choice questions showed a significant increase in knowledge in groups A–C and a nonsignificant one in group D (Wilcoxon signed rank test) (Table 4). The highest improvement was found in group B.
The scores for the free-form radiographic interpretation questions also showed a significant increase in groups A–C, with the highest improvement in group A (Table 5). Group D again showed no significant improvement.
|
Discussion
|
|---|
Many studies on educational interventions in radiology demonstrate limitations with respect to the number of students evaluated, bias due to low voluntary participation rates, and heterogeneous measuring tools (15–17). Owing to our ability to make the curricular intervention of this study compulsory for all students, a selection bias could be avoided: The 85% of the students (n = 192) included in our evaluation represents a high number and allows a powerful interpretation of the results. Nevertheless, we are aware of some limitations of our methodologic approach: We did not track the interpersonal communications between students in front of the screen or the light box, which might have been essential for a successful learning experience. All three intervention groups had the same chance for small-group interactions; therefore, the results of our measures should be equally biased in this respect. Furthermore, it seems ethically problematic, in general, to primarily exclude a group of students from an educational opportunity. We decided to make this sacrifice to obtain an adequate control group for comparison. Finally, since we conducted a short-term study over only 3 months, we were not able to answer questions on long-term effects like National Boards Examination results in radiology and organizational problems of curriculum planning. Long-term studies are even more difficult to control for confounding factors (15,24).
Media Comparison
Media comparison studies raise some general methodologic questions: It is quite difficult to control for confounding variables (25). Furthermore, the hypothesis for such a comparison should be clearly defined and not only a justification of the use of new media technology (20).
The design of our study does not allow comparison of case-based learning (interactive or noninteractive) with traditional learning by means of lectures and textbooks. We wanted to explore the effects of a small intervention with case-based learning performed with different technical approaches on acceptance, motivation, and learning success. We are not aware of any study on case-based learning in radiology in which an interactive computer-based approach was compared with a noninteractive computer-based approach. The studies of Erkonen et al (15) and D’Alessandro et al (24) compared a printed and a multimedia version of a radiology textbook in a non–case-based teaching scenario.
Motivation and Learning Success
The level of acceptance of case-based learning with or without computers was very high, as in many other studies (26): Ninety percent of all students recommended the cases to their peers. The noticeably high correlation between the self-evaluated improvement in knowledge of the students and the objective test results gives us a strong argument for the validity of student evaluations. The validity of student evaluations is a controversial issue in the discussion of curricular evaluation.
We conclude that interactivity was highly valued by the students to whom it was offered. The fact that only 42% of the students in group B missed interactivity may be due to the fact that there was no crossover between groups and therefore no direct comparison between the interactive and noninteractive approaches. The interactivity between learners in front of the screen may have affected the data and was not analyzed. The interpretation of these results is therefore difficult, and a direct controlled comparison with a crossover design should be performed in the future.
The case-based approach should teach the ability to connect clinical findings with imaging findings as an essential learning objective in radiology (27). Since it is generally difficult to evaluate learning success in problem-solving environments, we decided on a twofold testing scenario. Surprisingly, results from both the image interpretation test and the multiple-choice questions showed that the control group did not significantly improve its scores during the semester. On the other hand, all groups who were exposed to the cases showed significant improvement in both measures. Although no significant differences between the three intervention groups were detected (score differences; Wilcoxon signed rank test), it is noteworthy that the interactive computer-based group performed better in image interpretation and performed worse on the multiple-choice questions than did the noninteractive computer-based group. One might speculate that this observation becomes more prominent with a more fundamental intervention. For teaching the key ability of image interpretation in an applicable way, the interactive case-based format seems to offer clear advantages concerning the learning outcome. Multiple-choice questions on image interpretation seem to be less adequate to prepare students for clinical work.
Some well-appreciated studies from instructional psychology on case-based learning suggest that interactive teaching is superior to noninteractive teaching with respect to medical practice (28–31). The observation that the control group did not improve on the multiple-choice question test but showed a tendency toward improvement on the image interpretation test can be attributed to an internal medicine course in the same semester, which includes bedside radiographic interpretation and was attended by all students. The radiology lecture was obviously attended by only a small proportion of the students. The students who attended the lecture rated it subjectively as high as the case-based course. All groups used radiology textbooks with a relatively low level of intensity. Therefore, what was the reason for the significant differences in the test results? We conclude that the case-based intervention was the main reason, despite all of the confounding factors. The 3-hour intervention with 10 mini-cases on chest and bone radiology led to a significant increase in knowledge of radiology.
Implementation of Casebased Learning for Radiology
One can only speculate about the effects of a more complete radiology case library to support all key topics of the lecture. As more cases are being authored (including nuclear medicine objectives), we will soon be able to evaluate this scenario. On the basis of this study, one can confidently speculate that the effects will be positive.
The high usability of the CASUS authoring system (23) contributed to the efficacy of case creation. This interactive radiology case library can be used for easy dissemination on compact disk, read-only memory (CD-ROM) or via an intranet or the Internet. Our study did not include evaluation of the case-authoring process. However, the initial results of an ongoing evaluation are available. It seems to be advantageous to support the content experts with a student tutor (32).
The creation of paper cases used up more resources, including the costly generation of hundreds of film radiographs. When the respective computing infrastructure is available, we therefore recommend the more flexible and affordable computer-based use of cases.
Our cases were used in a self-study mode with the technical advice of a student tutor. We did not investigate the potentially beneficial effect of a radiologist who could answer further questions related to the case content. We believe that such a setting could make our approach even more useful (33).
In conclusion, the results of our study support the integration of computer-based cases for teaching radiology into the clinical medical curriculum as a supplement to lectures and practical courses. The continuation of implementation efforts may not need to focus on controlled comparisons. Instead, improvement of the integration process and development of content (20) will require special attention.
|
Footnotes
|
|---|
2 Both authors contributed equally to the work. 
|
References
|
|---|
-
Patton DD. Computer-assisted instruction in the radiological sciences using a desk-top computer. Radiology 1971; 100:553-559.[Medline]
-
Skinner JB, Knowles G, Armstrong RF, Ingram D. The use of computerized learning in intensive care: an evaluation of a new teaching program. Med Educ 1983; 17:49-53.[Medline]
-
Pickell GC, Medal D, Mann WS, Staebler RJ. Computerizing clinical patient problems: an evolving tool for medical education. Med Educ 1986; 20:201-203.[Medline]
-
Costaridou L, Panayiotakis G, Sakellaropoulos P, Cavouras D, Dimopoulos J. Distance learning in mammographic digital image processing. Br J Radiol 1998; 71:167-174.[Abstract]
-
Sinha S, Sinha U, Kangarloo H, Huang HK. A PACS-based interactive teaching module for radiologic sciences. AJR Am J Roentgenol 1992; 159:199-205.[Abstract/Free Full Text]
-
Desch LW. Use of commercial "authoring systems" for medical education. Med Educ 1986; 20:417-423.[Medline]
-
Murray TS, Barber JH, Dunn WR. Attitudes of medical undergraduates in Glasgow to computer-assisted learning. Med Educ 1978; 12:6-9.[Medline]
-
Brown DW, Groome DS, Niehoff RD, Cleaveland JD. Computer-assisted instruction in nuclear medicine. JAMA 1968; 206:1059-1062.[Abstract/Free Full Text]
-
Kuszyk BS, Calhoun PS, Soyer PA, Fishman EK. An interactive computer-based tool for teaching the segmental anatomy of the liver: usefulness in the education of residents and fellows. AJR Am J Roentgenol 1997; 169:631-634.[Abstract/Free Full Text]
-
Morin FD, Dubreuil B, Dussault RG, DiCori S, Bret PM. The radiographic signs of arthritis: a computer teaching module. RadioGraphics 1995; 15:703-708.[Abstract]
-
Keane DR, Norman GR, Vickers J. The inadequacy of recent research on computer-assisted instruction. Acad Med 1991; 66:444-448.[Medline]
-
Armstrong R. Interactive computer programs. J Rheumatol 1999; 26(suppl 55):56-57.
-
Jaffe CC, Lynch PJ. Computer-aided instruction for radiologic education. RadioGraphics 1993; 13:931-937.[Medline]
-
Schmidt HG, Dauphinee WD, Patel VL. Comparing the effects of problem-based and conventional curricula in an international sample. J Med Educ 1987; 62:305-315.[Medline]
-
Erkonen WE, D’Alessandro MP, Galvin JR, Albanese MA, Michaelsen VE. Longitudinal comparison of multimedia textbook instruction with a lecture in radiology education. Acad Radiol 1994; 1:287-292.[Medline]
-
D’Alessandro MP, Galvin JR, Erkonen WE, et al. An approach to the creation of multimedia textbooks for radiology instruction. AJR Am J Roentgenol 1993; 161:187-191.[Abstract/Free Full Text]
-
Chew FS, Stiles RS. Computer-assisted instruction with interactive videodisc versus textbook for teaching radiology. Acad Radiol 1994; 1:326-331.[Medline]
-
Brown RL, Carlson BL. Early diagnosis of substance abuse: evaluation of a course of computer-assisted instruction. Med Educ 1990; 24:438-446.[Medline]
-
Mangione S, Nieman LZ, Gracely EJ. Comparison of computer-based learning and seminar teaching of pulmonary auscultation to first-year medical students. Acad Med 1992; 67(10 suppl):S63-S65.[Medline]
-
Friedman CP. The research we should be doing. Acad Med 1994; 69:455-457.[Medline]
-
Calhoun PS, Fishman EK. Developing a computer-assisted instruction program: a process overview for the radiologist. RadioGraphics 1997; 17:1277-1291.[Abstract]
-
Mammone GL, Holman BL, Greenes RA, Parker JA, Khorasani R. Inside BrighamRAD: providing radiology teaching cases on the Internet. RadioGraphics 1995; 15:1489-1498.[Abstract]
-
Fischer MR, Schauer S, Grasel C, et al. CASUS model trial: a computer-assisted author system for problem-oriented learning in medicine. Z Arztl Fortbild 1996; 90:385-389[German].
-
D’Alessandro DM, Kreiter CD, Erkonen WE, Winter RJ, Knapp HR. Longitudinal follow-up comparison of educational interventions: multimedia textbook, traditional lecture, and printed textbook. Acad Radiol 1997; 4:719-723.[Medline]
-
Clarke R. Dangers in the evaluation of instructional media. Acad Med 1992; 67:820-821.
-
Jelovsek FR, Adebonojo L. Learning principles as applied to computer-assisted instruction. MD Comput 1993; 10:165-172.[Medline]
-
Norman GR, Brooks LR, Coblentz CL, Babcook CJ. The correlation of feature identification and category judgments in diagnostic radiology. Mem Cognit 1992; 20:344-355.[Medline]
-
Norman GR, Brooks LR, Cunnington JP, Shali V, Marriott M, Regehr G. Expert-novice differences in the use of history and visual information from patients. Acad Med 1996; 71(10 suppl):S62-S64.[Medline]
-
Regehr G, Norman GR. Issues in cognitive psychology: implications for professional education. Acad Med 1996; 71:988-1001.[Medline]
-
Mennin SP, Friedman M, Skipper B, Kalishman S, Snyder J. Performances on the NBME I, II, and III by medical students in the problem-based learning and conventional tracks at the University of New Mexico. Acad Med 1993; 68:616-624.[Medline]
-
Schmidt HG, Machiels-Bongaerts M, Hermans H, ten Cate TJ, Venekamp R, Boshuizen HP. The development of diagnostic competence: comparison of a problem-based, an integrated, and a conventional medical curriculum. Acad Med 1996; 71:658-664.[Medline]
-
Maleck M, Fleissner S, Fischer MR. Drag and drop authoring of cases for clinical teaching with CASUS. Do clinicians manage the technology and didactics alone? Med Teacher; (in press).
-
Schmidt HG, Moust JH. What makes a tutor effective?. A structural-equations modeling approach to learning in problem-based curricula. Acad Med 1995; 70:708-714.[Medline]
This article has been cited by other articles:
|
|
|
|
 
A. L. Zwingenberger and P. R. Ward
Medical Imaging Resource Center (MIRC) for Veterinary Medicine: A Digital Image Teaching File
J Vet Med Educ,
January 1, 2006;
33(4):
618 - 621.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Introduction
