DOI: 10.1148/rg.264055212
RadioGraphics 2006;26:1243-1249
© RSNA, 2006
Education Techniques for Lifelong Learning
A Novel Standard-Compliant Audience Response System for Medical Education1
Jonathan L. Streeter, MD and
Frank J. Rybicki, MD, PhD
1 From the Department of Radiology, Brigham and Women’s Hospital, 75 Francis St, Boston, MA 02115. Recipient of a Cum Laude award for an infoRAD exhibit at the 2004 RSNA Annual Meeting. Received December 15, 2005; revision requested February 8, 2006 and received February 27; accepted March 8. J.L.S. is founder and owner of BeamStat, a company founded to foster the development and distribution of education technology; F.J.R. has no financial relationships to disclose.
Address correspondence to J.L.S. (e-mail: jonstreeter{at}hotmail.com).
 |
Abstract
|
|---|
Audience response systems (ARSs) have been suggested as a means to improve the effectiveness of didactic lectures, but their widespread use has been limited because of cost and complexity. The authors developed a simple, cost-effective ARS that relies completely on standard networking technology, is embedded easily into a PowerPoint presentation, and operates without the need for extensive server setup and configuration. A PowerPoint Add-in that capitalizes on WiFi (wireless fidelity) technology was created that allows lecturers to easily add interactive question and response slides to an existing PowerPoint presentation. In the system, audience members can submit responses by using any wireless network-enabled device (eg, laptop computer, cell phone). The system was designed to be easy to use, to be standard compliant, and to eliminate the high operating costs associated with current interactive ARSs.
© RSNA, 2006
|
Introduction
|
|---|
The didactic lecture is a core component of medical education, serving as the primary tool by which educators build their students’ fund of knowledge in the medical sciences. Didactic lectures are also commonly used for postgraduate medical education (residency and continuing medical education), despite the fact that they have had variable effectiveness as measured by short- and long-term factual recollection (1) and modification of physician behavior (2). A student is more likely to recall information from an engaging lecture, and physician educators have employed various means of increasing the effectiveness of the traditional lecture format.
The use of an audience response system (ARS) has been suggested as a means to improve the effectiveness of didactic lectures and, in some cases, healthcare outcomes (1). The ARS is one method by which the lecturer can engage an audience, particularly a large audience, by incorporating questions and student interaction during the lecture. The lecturer poses a question to the audience and provides an array of multiple-choice answers. The audience members select their responses by using the ARS interface, and the results are displayed in real-time (Fig 1).
The use of an ARS has the advantage of allowing the lecturer to assess learning and reinforce key concepts as the audience responds anonymously in real-time.
View larger version (44K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1. Schematic illustrates an ideal ARS. Audience members equipped with a variety of wireless personal devices (lowest row of diagrams) submit their responses to questions posed by the lecturer (upper left), and the responses are displayed in graphic form in the context of the presentation.
|
|
Although ARS technology has been available since the 1970s, widespread utilization has been limited by the hardware cost and the complexity of preparation and maintenance of the equipment. We have developed a simple, cost-effective method to add an interactive component to didactic lectures. The method is based on a novel application and a PowerPoint Add-in (Microsoft, Redmond, Washington) to capitalize on modern wireless network standards (802.11 or WiFi [wireless fidelity] technology). The resulting system allows lecturers to add interactive question and response slides easily to existing presentations. It also allows audience members to submit responses by using any wireless network-enabled device (handheld or laptop computer, advanced or next-generation handheld video game console, or cell phone.)
To our knowledge, there is no ARS that relies completely on standard networking technology, is embedded easily into a PowerPoint presentation, and operates without the need for extensive server setup and configuration. The purpose of this article is to introduce and describe BeamStat, an ARS that meets these goals and thus provides an affordable method for routine implementation of audience response in medical education. In addition, we present a brief overview of ARSs and wireless technology.
|
History of ARSs
|
|---|
ARS technology has gone through several generations. Early systems used wired keypads, which made them costly and limited portability. Subsequent systems employed infrared wireless communication. Although this development improved portability, the system was limited because infrared dependence on "line of sight" and the lack of two-way infrared communication frequently led to incorrect tallying of responses. Radiofrequency-based systems overcame limitations of infrared systems, and the result was improved accuracy over a larger scale. For example, a current commercial ARS can process responses from over 2000 keypads and allows for participants to be 150 feet from the lectern. A main limitation for each of these original ARS implementations is that the technology depended on proprietary hardware, which represents a significant investment of money and resources to purchase and maintain such a system. Because of these costs, presentations that use ARS are typically reserved for large audiences for which the expense of leasing the hardware can be justified.
As the technology to communicate responses has evolved, so has the method to display audience results. Early systems required a display dedicated to showing the results, and the use of either a separate screen or a switch away from the lecture slides distracted from the content and flow of lectures. More recent systems have overcome this limitation, which enables responses to be incorporated directly into PowerPoint, a standard didactic lecture presentation tool.
Although standard Internet networking protocols and World Wide Web (WWW) technology is commonly used outside medical education (eg, to tabulate responses to opinion polls), ARSs have only begun to take advantage of standard, non-proprietary network communication.
The system introduced herein has the advantage that it is based on established Web protocols and thus allows the use of any WiFi-certified device (described in the next section, "Wireless Technology").
|
Wireless Technology
|
|---|
The IEEE (Institute of Electrical and Electronic Engineers) 802.11B wireless networking specification (3) was the first industry standard widely manufactured and adopted for public wireless network communication. Incorporation of IEEE 802.11B technology into a device allows any standard Internet protocol, including the hypertext transfer protocol (HTTP) used on the WWW, to be accessed without use of wires.
Compatibility between devices proved to be an early obstacle for IEEE 802.11B technology. Specifically, a device manufactured by one company often did not have reliable communication with a similar device manufactured by another company. These problems were successfully addressed by the creation of the WiFi alliance and certification. The WiFi alliance is a trade organization of over 200 technology companies; they established a WiFi certification for hardware that uses the IEEE 802.11B technology. Thus, all "WiFi-certified" devices, regardless of manufacturer, meet specific criteria for interoperability. The availability and utilization of these devices has grown exponentially, extending from laptop to handheld computers.
A typical wireless network setup would include one or more access points, each allowing multiple devices to connect. The maximum number of simultaneous users per access point is variable and depends on the manufacturer. A commercial access point, which typically costs less than $100, may support 30–250 users and would be ideal for a classroom or small group. An "enterprise-grade" access point, which typically costs less than $500, may support up to 1000 users. Enterprise wireless installations typically deploy multiple, redundant access points to ensure complete coverage and robust bandwidth support.
|
Implementation of a Simple ARS
|
|---|
BeamStat Application Components
ARSs have two essential functions: (a) the ability to receive audience input and (b) the ability to display the results in the context of the presentation.
The BeamStat ARS (Boston, MA) accomplishes these functions through the creation of a server component to handle the audience feedback and a PowerPoint module to allow insertion and display of interactive data.
The BeamStat server component acts as both an embedded Web server and database to allow and store input from network-enabled devices. The server component displays a Web page for responding when accessed by any network-capable device with a Web browser (Fig 2). Each wireless personal device is given a unique identifier; when the device’s operator selects an option, both the selection and unique identifier are stored locally. When called on during the presentation, the server component graphs the stratified audience response data with a selected PowerPoint slide.
View larger version (76K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 2. Screen capture of a wireless personal device (in this case a Macintosh laptop computer [Apple Computer, Cupertino, Calif]) shows a BeamStat Web page for answering questions in the ARS presentation. The audience members direct their Web browsers to the IP address provided to access BeamStat. A major advantage of BeamStat presentations is that audience members may use their own devices, which dramatically reduces the cost and complexity inherent in conducting an ARS presentation, compared with the resources needed for all other currently available ARSs.
|
|
Embedding a display of audience feedback within a PowerPoint slide is accomplished by adding and extending the functionality of Microsoft Office applications through the use of Visual Basic for Applications (VBA), a development technology and macro language. VBA allows creation of so-called add-ins to customize PowerPoint features, menus, and toolbars. Beam-Stat uses an add-in that was created to allow straightforward, seamless addition of audience feedback display within a PowerPoint presentation.
To ensure that all necessary components are easily installed and uninstalled on the presenter’s computer, both installation and setup programs as well as an uninstall feature are provided. No software installation is necessary on the audience devices. The system requirements to use the BeamStat software for the ARS require that the presentation computer be running Windows XP (Microsoft) and PowerPoint 2000 or higher.
Software Installation
The user must install the BeamStat software (free download from www.BeamStat.com) before preparing or delivering the presentation. Once the program is downloaded, the user can double-click on the installer file to create the program folder and place the BeamStat icon on the user’s desktop. In addition, the BeamStat software adds new functions to the PowerPoint "Insert" menu that allow insertion of an interactive slide. If the computer used to deliver the presentation differs from the one used to create the presentation, BeamStat must be installed on both computers to allow the interactive functionality.
Adding Interactive Functionality to a Question and Answer Slide Pair
After successful installation of the BeamStat software, two additional items appear on the Power-Point "Insert" menu: "Insert BeamStat" and "Insert Instruction Page." Adding interactive functionality to a PowerPoint slide is then accomplished by following the next three steps.
- The user creates a new slide or moves to an existing slide within a presentation that contains a question to be posed to the audience.
- The user selects "Insert BeamStat" from the Insert menu. A wizard, or step-by-step guide, appears to assist the user through the process of inserting a graph to display the results of the question (Fig 3).
- The user follows the instructions in the wizard to customize the content and display of the graph.
After completing the wizard, the software first inserts a small counter placeholder in the lower right-hand corner of the question slide. Next, a new slide is added with a display graph place-holder. The user may then customize both the question slide and the graph slide to fit the context of the question and display graph. During editing, no data are displayed, and a default Windows icon is shown as a placeholder. The placeholder may be resized and moved to suit the user’s design. The resulting counter and graph are resized and moved accordingly. This procedure is repeated for each interactive question in the presentation.
Inserting the Instruction Slide
An instruction slide is used at the beginning of each presentation to teach audience members where to direct their Web browsers. This slide includes the unique identifying number, or Internet protocol (IP) address, of the computer used for the presentation.
To insert the instruction slide, with the Power-Point presentation open for editing, the user selects the first slide in the presentation and then chooses the "Insert Instruction Page" option from the Insert menu. A new slide containing a placeholder for the instruction page is created. When the presentation is run, this page will display instructions for the audience that include the IP address of the presentation computer that will be used for the responses.
It is recommended that the instruction slide be displayed while the audience prepares for the presentation; this preview allows each member of the audience to ensure his or her connectivity. In addition, the server IP address should be added to the master slide, either as a header or footer. This information will allow participants who arrive after display of the Instruction page to know the IP address for their Web browser.
Running the Server
The instructor starts the BeamStat server before beginning the presentation. The server component runs in the background and receives input from the audience members, eventually transferring these data to a PowerPoint slide for display. To start the server component, the user selects the icon from the desktop or start menu. Once activated, an icon indicating the server status is placed in the system tray of the Windows XP taskbar.
The server component functions as an embedded dedicated Web server and does an initial check to verify its ability to communicate with other computers. After communications have been verified, the server icon turns green, indicating that it is ready to collect responses and relay them to the PowerPoint presentation. As soon as the server has been activated, audience members who enter the server’s address in their Web browser will be presented with the input screen (Fig 4). No answers will actually be recorded, however, until the appropriate question slide is viewed during the presentation.
View larger version (112K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 4. Photograph shows the BeamStat audience input screen as viewed from various personal handheld devices. The flexibility of BeamStat enables the collection of audience feedback from a wide variety of transmitting devices, increasing its potential utilization outside medical education.
|
|
With the server component activated and a green light displayed in the system tray icon, the interactive presentation can be started.
Performing the Presentation
Once the BeamStat server component is running, the PowerPoint presentation is started by selecting "View Show" from the Slideshow menu. The presentation is advanced in the usual manner until a question slide is reached. The instructor then poses the question to the audience members and asks them to key in their responses. The counter provides real-time display of the number of audience members who have replied (Fig 5a). Once the presenter is satisfied with the number of answers received, the presentation is advanced to the next slide in which the replies are presented in bar graph form (Fig 5b).
View larger version (62K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 5a. (a) Screen capture of a PowerPoint question slide illustrates how the lecturer and audience can view updated responses in real-time. The counter in the bottom right-hand corner displays the number of audience members who have replied to a question. (b) Screen capture of a PowerPoint answer slide displays the stratified results of the audience feedback in graphic form.
|
|
View larger version (59K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 5b. (a) Screen capture of a PowerPoint question slide illustrates how the lecturer and audience can view updated responses in real-time. The counter in the bottom right-hand corner displays the number of audience members who have replied to a question. (b) Screen capture of a PowerPoint answer slide displays the stratified results of the audience feedback in graphic form.
|
|
Reviewing Statistics
Audience response data can be stored for later analysis. At the completion of a presentation, the BeamStat server component formats the audience data as a Microsoft Excel spreadsheet and stores the file in the BeamStat program folder on the user’s hard drive.
|
Discussion
|
|---|
Although data that demonstrate the effectiveness of ARSs are limited, Schackow et al (1) found that the use of an ARS by a small number of family practice medical residents improved learning outcomes. These students demonstrated significantly improved immediate factual retention, as well as relatively greater knowledge retention a month after intervention. These preliminary results in medical education mirror data from other educators. For example, a survey of 6000 students in an introductory physics course that uses ARS revealed that ARS increased effectiveness of the learning beyond that obtained from traditional didactic lectures (4).
In medicine, ARS methods have been used to collect data regarding the effectiveness of resident education. Eggert et al (5) found that use of an ARS facilitated and documented changing audience perception during a periodic clinical decision-making journal club. The use of an ARS quantified convergence of audience opinion on a single answer when audience responses were compared before and after evidence supporting a clinical question was presented.
Higher learning institutions have attempted to promote digital technology as an educational resource by providing laptop computers to their students. An early (initiated in 2002) and ambitious example is the Maine Learning Technology Initiative (http://www.maine.gov/mlte/), which has a goal of providing a laptop computer to every seventh and eighth grade student and teacher in the state of Maine. Michigan, New Hampshire, New Mexico, and Pennsylvania are all developing similar programs. Many institutions of higher education both within healthcare and other industries also provide personal digital devices.
Instructors are learning, however, that computers and Internet access have the potential to become a digital distraction in the classroom. The January 2004 USA Today article "High tech creates new classroom distractions" (http://www.usatoday.com/tech/news/2004-01-20-blackboard-blues_x.htm) describes challenges faced by instructors at institutions that provide laptops and classroom Internet access. In addition to its application in medical education, we hope that Beam-Stat can be used as a tool to focus and monitor the attention of members of a classroom.
(Correctly employed, BeamStat may optimize an institution's investment in technology and reduce classroom distraction.)
Although its unique use of standard wireless networking reduces the cost of implementing an ARS, a BeamStat presentation requires that each responding member of the audience be equipped with a network-enabled device. This requirement carries some financial burden, but the overall cost is already significantly less than that of dedicated ARS hardware, and the price of low-end WiFi devices continues to drop.
Although the standard-compliant nature of the method employed to collect audience feedback allows for great flexibility in transmitting devices, at present the computer used to deliver the presentation must run Windows XP and PowerPoint 2000 or higher. Support for lectures given using the Macintosh operating system platform and Keynote presentation software (Apple Computer) is currently under development.
Network security remains an important concern. In a highly secure environment (ie, where network access is rigorously protected), the presenter should consult with the information technology department. Although BeamStat represents no greater security threat than simple use of a Web browser, additional security measures for wireless network access may require special permission. In addition, it is possible to run a Beam-Stat presentation by using a wireless router that is connected only to the presenter’s computer and not connected to either the hospital network or the Internet. This setup allows all WiFi-enabled devices in the immediate area to communicate with each other and is sufficient for interactive functionality and represents no security threat to the institution.
It is possible to receive input from devices that are Internet capable but do not use WiFi as their networking technology. Examples include a Web-enabled cell phone or a Blackberry device (Research in Motion, Waterloo, Ontario). Network configuration that allows such users to submit responses is potentially more complex if certain network security measures, such as a firewall, are in place. There is no fundamental limitation of using these devices. However, proper network configuration is required for interactive capability.
Finally, as is the case when any new method is introduced in medical education, there is a learning curve to achieve optimal utilization. New users should consult established guidelines for interactive presentation (6) to minimize common mistakes. Tips include limiting the number of choices presented for each question, using interactive features mainly for key points, and limiting the complexity of the questions. Experience with the anticipated flow of the presentation and interspersed feedback questions will allow the use of more complex and dynamic interactions to improve the quality of medical education.
|
Conclusions
|
|---|
BeamStat provides a unique, simple, and cost-effective ARS to add an interactive component to didactic lectures.
Audience members may submit responses by using any wireless network-enabled device. BeamStat is designed to be both easy to use and standard compliant, and it is hoped that the technology can enrich the content of didactic lectures in medical education.
|
Acknowledgments
|
|---|
The authors acknowledge the contributions of the team of developers, William Nalen, BS, Michael Sharp, BA, and Jason Smith, who assisted in creating the BeamStat application; Spencer Streeter, who provided illustrations and graphic design; and Kitt Shaffer, MD, PhD, who provided support for an early prototype.
|
Footnotes
|
|---|
Abbreviations: ARS = audience response system, IEEE = Institute of Electrical and Electronic Engineers, IP = Internet protocol, WiFi = wireless fidelity
Editor’s Note: The BeamStat wireless Audience Response System is available without cost to groups and classrooms of up to 20 audience members.
|
References
|
|---|
- Schackow TE, Chavez M, Loya L, Friedman M. Audience response system: effect on learning in family medicine residents. Fam Med 2004;36:496–504.[Medline]
- Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282:867–874.[Abstract/Free Full Text]
- Wireless LAN medium access control (MAC) and physical layer (PHY) specifications: higher-speed physical layer extension in the 2.4 GHz band. IEEE-SA Standards Board. Available at: http://standards.ieee.org/getieee802/download/802.11b-1999.pdf. Accessed September 10, 2005.
- Hake RR. Interactive engagement versus traditional methods: a six-thousand student survey of mechanics test data for introductory physics courses. Am J Phys 1998;66:64–74.
- Eggert CH, West CP, Thomas KG. Impact of an audience response system. Med Educ 2004;38:576.[CrossRef][Medline]
- Robertson LJ. Twelve tips for using a computerized interactive audience response system. Med Teach 2000;22:237–239.
Top
Abstract
Introduction
