(Radiographics. 2000;20:279-286.)
© RSNA, 2000
The Visible Man: Three-dimensional Interactive Musculoskeletal Anatomic Atlas of the Lower Extremity1
Heung Sik Kang, MD,
Bo Hyoung Kim, MS,
Jae Wook Ryu, MD ,
Sung Hwan Hong, MD,
Hye Won Chung, MD,
So Yeon Cho, MD,
Young Hoon Kim, MD ,
Sung Il Hwang, MD,
Dong Kyun Jeong, BA and
Yeong Gil Shin, PhD
1 From the Department of Radiology, Seoul National University College of Medicine, 28 Yongon-Dong, Chongno-Gu, Seoul 110-744, Korea (H.S.K., J.W.R., S.H.H., H.W.C., S.Y.C., Y.H.K., S.I.H.); the Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (H.S.K., J.W.R., S.H.H., H.W.C., S.Y.C., Y.H.K., S.I.H.); and the Department of Computer Science, Seoul National University, Korea (B.H.K., D.K.J., Y.G.S.). Presented as an infoRAD exhibit at the 1998 RSNA scientific assembly. Received March 1, 1999; revision requested May 12 and received July 14; accepted July 26. Supported by grant HMP-98-G-1-002-A from the 1998 Highly Advanced National Projects on the Development of Biomedical Engineering and Technology. Address reprint requests to H.S.K.
 |
Abstract
|
|---|
A personal computer–based interactive musculoskeletal anatomic atlas of the lower extremity has been created by using the Visible Human Male data set. A semiautomatic segmentation program was developed by using an intelligent scissors approach and shape-based interpolation, thus considerably reducing the laborious work of the segmentation and labeling process. Manual contour extractions at 3-mm section intervals and shape-based interpolations of intervening sections of the musculoskeletal structures of the lower extremity were performed. For interactive and realistic three-dimensional display, an efficient binary volume rendering method was developed that introduces the concept of shear-warp factorization and applies a newly developed normal calculation technique. Binary volume rendering reconstructs various structures from a series of two-dimensional sections in a few seconds, thus enabling real-time manipulations of the computerized atlas. All of the muscles, tendons, and bones of the lower extremity have been segmented and labeled. The volume-based three-dimensional interactive atlas supports various interactions including rotation, removal, highlighting with artificial colors, arbitrary cutting operation, transparent view, and descriptive knowledge representation. In addition, browsing through the two-dimensional images of transverse, coronal, and sagittal views with labeling and segmentation information is possible.
Index Terms: Computers, educational aid • Extremities, 40.92 • Images, display • Images, three-dimensional
|
Introduction
|
|---|
Recent advances in computer-related technology allow production of interactive three-dimensional (3D) atlases of human anatomy (1–6). Three-dimensional visualization is helpful in understanding human anatomy by presenting the information in a form that is not only pleasing but also easily recognizable. The advantages of a computerized anatomic atlas include the ability to support various interactive approaches to anatomic information and education. Interacting directly with anatomic structures contributes to a more comprehensive understanding of them in spatial relation to their surroundings. Unlike a conventional anatomic atlas, a computerized anatomic atlas can also be an ideal complement to conventional cadaveric dissection. In addition, 3D reconstruction of anatomic structures provides a virtual environment that can facilitate surgical planning, virtual surgery, virtual endoscopy, and training simulations (4–10).
The Visible Human Project data set of the National Library of Medicine (11), the most complete computerized database of the human body ever assembled, enables the creation of 3D models of the human body based on cross-sectional images. The high-resolution data of the Visible Human Male is an ideal basis for a computerized atlas (3,6,8). Considerable progress has been made in the fields of segmentation, navigation, simulation, modeling, visualization, and education. However, most projects based on the Visible Human data set have focused on the brain and major visceral organs (3,7,8,10).
Although the musculoskeletal system, with its complex osseous and soft-tissue anatomy, is suitable for a computerized anatomic atlas (12–14), an interactive 3D anatomic atlas of the entire musculoskeletal system has not been developed, to our knowledge. Difficulty with segmentation is the most important obstacle to producing an interactive musculoskeletal anatomic atlas. Muscles are easily segmented due to their prominent color; however, tendons are difficult to segment because of their reduced color difference from that of surrounding fatty tissue and their small size. Also, segmentation of musculoskeletal structures requires much laborious work because more than 1,800 sections of the Visible Human Male data set contain musculoskeletal structures.
We have produced a personal computer–based interactive musculoskeletal anatomic atlas from the Visible Human Male data set. The atlas was created by applying a semiautomatic segmentation method, which considerably reduces the laborious work of the segmentation process, and a volume rendering method that allows interactive visualization of color volume data. This article describes the development of the atlas, including data acquisition, segmentation and labeling of anatomic structures, 3D reconstruction, and user interactions.
|
Data Acquisition
|
|---|
We used the digital photographs of cryosections of the Visible Human Male. The cross-sectional images were produced with a pixel resolution of 0.33 mm and a section distance of 1 mm. The image pixel resolution was reduced from 0.33 mm to 1 mm by averaging 3 x 3 neighboring pixels to obtain a regular volume. For a real 3D atlas, the structures of the human body should be identified by means of segmentation and labeling. A variety of structures including muscles, bones, and tendons were separately extracted with semiautomatic segmentation. The data obtained with the above steps were converted to a specific data structure appropriate for 3D visualization. The overall data acquisition steps for our atlas are shown in Figure 1.
|
Segmentation and Labeling of Anatomic Structures
|
|---|
The Visible Human images are presented in a series of transverse sections (11). Development of a 3D interactive atlas requires detailed segmentation of the transverse images. In image analysis, the term segmentation refers to subdivision of an image or data voxel into its constituent parts.
Segmentation and labeling of cross-sectional images require specialized anatomic knowledge. For this purpose, color printing of each transverse section was first performed. Then, radiologists (J.W.R., S.H.H., S.Y.C., Y.H.K., S.I.H.) drew the contour lines of structures including bones, muscles, and tendons on the printed images. Final segmentation information was obtained by using a pencil digitizer and a semiautomatic program.
The intelligent scissors approach (15) was used for a substantial portion of the segmentation. Intelligent scissors is an interactive segmentation tool that formulates boundary specifications by means of graph searching with the goal of finding the optimal path between a start node and a set of goal nodes. Figure 2 shows the user-friendly interface of our segmentation program. This program provides direct boundary specification by means of mouse or pen actions. While the user contours the boundary of the structure being segmented, the program locates the pixels with the highest possibility of being the border of the structure according to the metrics of the intelligent scissors approach. In general, the segmentation process involves manual or semimanual separation and identification of organs. To reduce such inefficiencies, shape-based interpolation (16) was applied for semiautomatic segmentation. In this article, the term semiautomatic segmentation refers to estimation of the shape of an organ from widely spaced sections. Manual segmentation and labeling are performed at specified section intervals, and intervening sections are automatically generated with shape-based interpolation.
View larger version (79K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 2. User interface of the segmentation tool with all of the musculoskeletal structures in section 1,000 (thigh level) segmented and labeled. A = supporting tools, B = segmented transverse image, C = number of the segmented section, D = names of structures that can be labeled, m = muscle.
|
|
To date, we have segmented and labeled all of the muscles, tendons, and bones of the lower extremity (Fig 2). Segmentation of the skin was performed at 1-mm intervals. Other structures were segmented at 3-mm intervals, then intersection contour information was automatically generated with shape-based interpolation.
|
Three-dimensional Reconstruction
|
|---|
After segmentation and labeling of structures on the cross sections, 3D reconstruction was performed to create a computerized anatomic atlas. For 3D reconstruction, we treated the segmented volume as a binary volume consisting of only two density values (0 for inside and 1 for outside of a structure) and used a binary volume rendering technique. Our rendering method takes advantage of the strengths of the shear-warp factorization of a viewing matrix and a modified run-length encoding data structure that stores only the boundary voxels of a structure.
Our rendering method consists of depth-based binary volume rendering. This method determines a pixel value in the image plane by using the gradient on each surface comprising the voxel projected to that pixel. The gradient can be estimated by investigating the depth values of that pixel and its neighborhood. The quality of images generated with this method is not as good as that of images generated with volume ray casting. The rendering method should regenerate depth information and recalculate gradients whenever the viewing direction is altered. However, our method has several advantages: (a) It does not require additional storage for maintaining intermediate results. (b) It can generate moderate-quality images. (c) It can produce tolerable images even when the sampling rate of volume data is quite low.
To improve the speed of depth calculation, we devised a section-based depth calculation method (17) based on shear-warp factorization (18,19). A depth buffer is generally produced by first projecting the volume parallel to the viewing direction into the plane perpendicular to the viewing direction and assigning the distance value of the voxel visible from the viewpoint to a pixel value. Our method does not require such a distance buffer but does require an intermediate depth buffer that contains the section number of the valid voxel first encountered by a viewing ray.
Our binary rendering method consists of two phases (Fig 3). In the first phase, the shear and projection phase, the volume data is projected into a two-dimensional (2D) intermediate image in sheared object space, producing intermediate depth, color, and label images. By using the resulting depth and color images, color shading is performed in intermediate image space. In the second phase, the warping phase, the intermediate shade and label images are transformed to final image space.
The intermediate depth image contains the coordinate of the first-hit voxel along the axis most perpendicular to the viewing direction. The intermediate color image maintains color information for shading. The pixel value of the image is the red, green, and blue values of the valid voxel that a viewing ray intersects first. Our rendering method reduces the total rendering time by performing the shading in intermediate image space rather than final image space (17).
Our computerized atlas provides not only the simple 3D reconstruction but also descriptive knowledge such as labeling information in the 3D rendered image. Two label images, the intermediate and final label images, are used for the 3D referencing operation. The intermediate label image contains the labeled name of visible voxels from a given viewpoint in intermediate image space; this name is transformed to final image coordinates. When the mouse button is clicked on some area of the 3D rendered image, the color of the structure corresponding to the clicked area changes to a conventional color and its labeled name shows up.
Even if the reconstruction time is heavily dependent on the number of rendered structures, our rendering algorithm produces a frame with dozens of structures in a few seconds. Our performance results are summarized in Table 1. The timings were measured on a personal computer with a 400-MHz Pentium II processor (Intel, Santa Clara, Calif) and a 256-Mbyte random-access main memory.
In the 3D reconstruction images, the blue seamline artifacts at the thigh are due to the fact that original photographs affected by discoloration were used during the compensation of missing sections. The contour of each individual structure cannot be exactly identified by just using the intelligent scissors approach; the 3D reconstructions may be more or less unsatisfactory from a procedure-based viewpoint. However, they are sufficient for understanding the shapes of anatomic structures and interpreting the topological relationships between the structures.
|
User Interactions
|
|---|
Our anatomic atlas provides a user-friendly interface for user interaction and manipulation. Currently supported interactions are classified into two categories: 3D visualization with manipulation and 2D image browsing with labeling information (Table 2).
Figure 4 shows the user interface of the anatomic atlas. Arrow toolbar buttons are used to rotate the rendered structures and examine them in a different view direction. Three-dimensional reconstruction of the selected structures and rotation of the structures facilitate the understanding of anatomically complicated structures (Fig 5).
View larger version (44K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 4. User interface of the musculoskeletal anatomic atlas with results of rendering all segmented bones and tendons and one muscle (sartorius). Various user interactions are supported by user-friendly toolbars. Clicking the right mouse button anywhere within a 3D rendered structure produces a pop-up window of operations applicable to that structure.
|
|
The "coloring" operation is used to convert the color of the structure indicated with the mouse into an arbitrary color. With this operation, selected muscles can be highlighted with artificial colors (Fig 6). This operation allows us to add highlighting capability to the illustrative atlas and enhance the reality of the anatomic atlas as well.
View larger version (52K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 6. Highlighting with artificial colors and transparency effect. Volume-rendered images of all segmented structures show highlighting of the sartorius, rectus femoris, vastus medialis, and vastus lateralis muscles with artificial colors before (left) and after (right) translucent skin clothing.
|
|
The "clothe skin" operation overlays a transparent whole-body skin on 3D rendered structures (Fig 6). The transparency effect is obtained by managing the color information of a voxel as a four-channel value (red, green, blue, and alpha) and applying the alpha blending technique of traditional graphics (20). The alpha factor of structures that are already rendered and the alpha factor of the body skin can be adjusted by the user for an appropriate transparency effect. This transparency effect, which adds a new dimension to the human anatomic atlas, allows selected structures to be rendered translucent so that their orientation with respect to normally obscured structures can be understood.
Our computerized atlas supports interactive referencing of the labeled information in a 3D reconstruction. If one clicks the left mouse button within a certain area in a 3D rendering, the color of the structure corresponding to that area changes and the labeled name shows up (Fig 7).
View larger version (94K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 7. Interactive 3D referencing. Volume-rendered images of all segmented structures show interactive referencing of the labeled information for the sartorius and tibialis anterior (ant) muscles before (left) and after (right) translucent skin clothing.
|
|
An additional feature of our atlas is the ability to navigate through 2D images section by section with labeling information attached. Navigation through 2D transverse, coronal, and sagittal images is available. A 2D browsing window contains two images: a small, whole-body, thumbnail image with a horizontal or vertical line indicating the current browsing location and the currently browsed 2D image. Clicking the left mouse button anywhere within the 2D browsed image shows the labeled name of the structure represented by that area. The structure stands out in a conventional color along with the labeling information (Fig 8). Such 2D navigation is a valuable aid in understanding human anatomy for those who are not familiar with cross-sectional images.
View larger version (82K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 8a. Browsing through 2D images with interactive referencing. (a) Transverse image shows the gluteus medius muscle selected for interactive referencing. (b) Reconstructed coronal image shows the semimembranosus muscle selected for interactive referencing. (c) Reconstructed sagittal image shows the soleus muscle and Achilles tendon selected for interactive referencing.
|
|
View larger version (88K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 8b. Browsing through 2D images with interactive referencing. (a) Transverse image shows the gluteus medius muscle selected for interactive referencing. (b) Reconstructed coronal image shows the semimembranosus muscle selected for interactive referencing. (c) Reconstructed sagittal image shows the soleus muscle and Achilles tendon selected for interactive referencing.
|
|
View larger version (71K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 8c. Browsing through 2D images with interactive referencing. (a) Transverse image shows the gluteus medius muscle selected for interactive referencing. (b) Reconstructed coronal image shows the semimembranosus muscle selected for interactive referencing. (c) Reconstructed sagittal image shows the soleus muscle and Achilles tendon selected for interactive referencing.
|
|
|
Conclusions
|
|---|
We have developed a 3D interactive musculoskeletal anatomic atlas of the lower extremity based on the Visible Human Male data set. This volume-based 3D interactive atlas supports various interactions including rotation, removal, highlighting with artificial colors, arbitrary cutting operation, transparent view, and browsing through the transverse, coronal, and sagittal 2D images with labeling and segmentation information. Semiautomatic segmentation with the intelligent scissors approach and shape-based interpolation substantially reduces the tedious work of segmentation and labeling. Binary volume rendering reconstructs various structures from a series of 2D sections in a few seconds, thus enabling real-time manipulations of the computerized atlas.
|
Footnotes
|
|---|
Abbreviations: 2D = two-dimensional
3D = three-dimensional
|
References
|
|---|
-
Schubert R, Bomans M, Hohne KH, et al. A new method for representing the human anatomy. Comput Med Imaging Graph 1993; 17:243-249.[Medline]
-
ter Haar Romeny BM, Zuiderveld KJ, Van Waes PF, et al. Advances in three-dimensional diagnostic radiology. J Anat 1998; 193:363-371.
-
Toh MY, Falk RB, Main JS. Interactive brain atlas with the Visible Human Project data: development methods and techniques. RadioGraphics 1996; 16:1201-1206.[Abstract]
-
Whiten S, Smart SD, McLachlan JC, Aiton JH. Computer-aided interactive three-dimensional reconstruction of the embryonic human heart. J Anat 1998; 193:337-345.
-
Vanier MW, Marsh JL. Three-dimensional imaging, surgical planning, and image-guided therapy. Radiol Clin North Am 1996; 34:545-563.[Medline]
-
Gingins P, Beylot P, Kalra P, et al. Modeling using the Visible Human dataset; Available at: http://miralabwww.unige.ch/ARTICLES/MIE96B.html
..
-
Ross MD. 3-D imaging in virtual environment: a scientific, clinical, and teaching tool; Available at: http://www.nlm.nih.gov/research/visible/vhp–conf/ross/index.htm..
-
Hoffman H, Irwin A, Prayaga R, Danks M, Murray M. Virtual anatomy from the Visible Man: creating tools for medical education; Available at: http://www.nlm.nih.gov/research/visible/vhp–conf/hoffman/vhp–pape.htm..
-
Kerr JP, Sellberg M, Ratiu P, Knapp D, Caon C. Photorealistic volume rendered anatomical atlases and interactive virtual dissections of the Dissectable Human; Available at: http://www.nlm.nih.gov/research/visible/vhp–conf/kerr/nilpaper.htm..
-
Banvard RA, Ackerman MJ. The Visible Human Project conference; Available at: http://www.nlm.nih.gov/research/visible/vhp–conf/vhpconf.htm..
-
Spitzer V, Ackerman MJ, Schersinger AL, Whitlock D. The Visible Human Male: a technical report. J Am Med Inform Assoc 1996; 3:118-130.[Abstract/Free Full Text]
-
Lorenson W. Marching through the Visible Man In: Proceedings of the Visualization Conference. Piscataway, NJ: Institute of Electrical and Electronics Engineers, 1995; 368-373.
-
Tiede U, Schiemann T, Hohne KH. Visualizing the Visible Human. IEEE Comput Graph Appl 1996; 16:7-9.
-
Chan WP, Lang P, Chieng PU, Davison PA, Huang SC, Genant HK. Three-dimensional imaging of the musculoskeletal system: an overview. J Formos Med Assoc 1991; 90:713-722.[Medline]
-
Mortensen EN, Barrett WA. Intelligent scissors for image composition In: Proceedings of the Computer Graphics Conference. New York, NY: Special Interest Group on Computer Graphics of the Association for Computing Machinery, 1995; 191-198.
-
Herman GT, Zheng J, Bucholtz CA. Shape-based interpolation. IEEE Comput Graph Appl 1992; 13:69-79.
-
Kim BH, Lee CH, Shin YG, Kang HS. 3D reconstruction of color volume data. J Korean Soc Med Biol Eng 1998; 19:199-208.
-
Ylä-Jääski J, Klein F, Kubler O. Fast direct display of volume data for medical diagnosis. CVGIP: Graph Models Image Processing 1991; 53:7-18.
-
Lacroute P, Levoy M. Fast volume rendering using a shear-warp factorization of the viewing transformation. Comput Graph 1994; 28:451-458.
-
Porter T, Duff T. Compositing digital images. Comput Graph 1984; 18:49-55.
Top
Abstract
Introduction
