There's no question that computer visualization has revolutionized the field of medical imaging. The ability to generate 3D volumetric models of human anatomy based on data acquired through one of a number of imaging technologies provides physicians with an invaluable perspective on their patients' inner workings.
These visualizations, which are used for such purposes as treatment planning and physician education, typically rely on reconstructions of sequential 2D image "slices" of patient anatomy acquired either through computed tomography (CT) or magnetic resonance imaging (MRI). These imaging techniques have long been favored because they produce relatively distinct, clean pictures, which eases the digital reconstruction of the 3D volume.
An automatic culling mechanism rejects irrelevant pixels in an acquired image, alleviating some of the noise and fuzziness. Here, an ultrasound volume is depicted before and after the culling process. The top two images show the ultrasounds being registered. Third is an example of their registration with no culling; fourth, one using the culling technique. The fifth image shows the difference between the top two images. The bottom shows the difference between the first and fourth images.
For years, imaging specialists have been seeking ways to obtain similar results from other im aging meth ods, particularly ultrasound, which is an inexpensive, more read ily available, and risk-free im aging method. Un for tunately, despite its benefits, ultrasound data is rarely used as the basis for creating digital 3D models, primarily because it produces noisy, fuzzy pictures.
With an eye toward leveling the playing field among imaging technologies in this arena, visualization experts at an Israeli medical simulation company called MedSim Ltd., in conjunction with researchers at the University of Tel-Aviv, have developed a 3D visualization system meant to exploit the benefits of ultrasound imaging while alleviating the hallmark noise and deformations associated with the technology.
The method automatically aligns and registers pairs of ultrasound scans acquired from slightly different perspectives, but which have a significant overlap, then transforms them using a technique akin to morphing in order to create a single volume that smoothly combines the information from both. Groups of these transformed volumes are then stitched together in a mosaic in order to represent a large scanned area. "It's impossible to image large portions of the body with a single ultrasound scan be cause of the limitations of conventional ultrasound devices. The best one can do is to scan small volumes one at a time and combine them into a larger one," says project collaborator Daniel Cohen-Or, a computer science professor at the Uni versity of Tel-Aviv.
With CT and MRI data, such alignment and registration is often achieved using a feature-based registration method, whereby distinctive features (such as boundaries and edges) of scanned im ages are identified either au tomatically or manually and aligned using some sort of data interpolation or warping technique.
Such an approach is ill-suited to ultrasound data, however, because 3D features in an ultrasound can appear significantly different in two acquired images, even if the perspectives from which they were acquired are very close. Such variations can be attributed to many sources, including those related to viewpoint changes, signal values, and inevitable geometric distortions. For example, the ultrasound probe itself casts a shadow on the image, so when the probe viewing direction changes, so does the nature and location of the shadow. In addition, the ultrasound signal values themselves are variable. The resulting noise and blurring effects can occlude distinct features. Finally, body movement during the data-acquisition process (either related to the pressure and contact of the ultrasound probe or to the natural behavior of the sampled organ, such as a beating heart) can cause inconsistent geometric deformations.
To deal with the inherent variability of ultrasound data, the system developed by researcher Dror Aiger of Med Sim uses a unique im age-based registration technique that looks directly at the gray values of image pixels rather than attempting an interpretation of those values, as is the case with feature detection. The method compares and registers the volumes based on the information contained within each pixel. Because some of the pixel information in each image carries no useful information (arbitrary noise, errant shadows, and so forth), the technique re lies on a "majority" approach, in which the gray values of the pixels are considered in the context of the whole image rath er than independently. Therefore, the success of the registration reflects the average success over the whole image.
The gradient-registration technique is based on the assumption that the pairs of volumes being con sidered are close enough for the mapping be tween the two to be expressed in terms of their gray values.
Getting "close enough" images is a function of the data-acquisition process. A good approach for achieving this, according to Aiger, is to attach the ultrasound probe to a mechanical motor that sweeps the slice along a trajectory. For example, a parallel sweep produces a series of parallel slices with no interslice gaps. Such dense slices provide small volumes of good quality.
A volumetric mosaic of a weeks-old fetus was developed by registering overlap regions of ultrasound scans based on correlations of gray values among the collected images. This is a cross-section of the mosaic.
Once the data has been acquired, the gradient-registration technique aligns and registers it using a two-staged process. The first step, called a coarse rigid transformation, registers one volume to another based on pixel correlations in the overlap portion of the volumes. To deal with larger-than-ideal gaps between the slices, a resolution pyramid enables the mapping between the volumes to be expressed in terms of their gradient values for coarse to fine resolution. The resolution pyramid comprises the original image and copies of it at increasingly lower resolutions. At lower resolutions, adjacent pixels and local gradients represent large distances between the target and the source images. Smaller gaps enable higher resolution.
The gradient-registration technique also incorporates a culling mechanism that eliminates from consideration all the pixels that don't have a matching counterpart in the other image or that are deemed by the system to be void of useful information based on statistical data from the pixel's neighborhood. Removing pixels that are irrelevant to the solution significantly improves the registration of partially occluded or generally noisy scenes.
The second step in the transformation process is a fine-elastic warp that extends the gradient registration to 3D and warps the data based on the elastic content (from body movements) of the volume.
To create the 3D volume representation, the newly warped volume is "stitched" over the reference volume. The size of the ultimate mosaic is limited only by the main memory of the workstation on which it's being composed and generally depends on the application for which the visualization is intended. For example, MedSim has patented the technology for use in a real-time ultrasound-based visual simulator that it has developed for medical training. With the system, students learn how to identify and diagnose a range of medical conditions by operating a simulated ultrasound device on a mannequin. For this application, says Cohen-Or, "the mosaics let us build a database large enough for training purposes."
In addition to the ultrasound simulator, other medical ultrasound applications of the technology include the development of a 3D anatomy atlas, says Cohen-Or. "It can also be used for the analysis of large organs, where a diagnosis has to be based on an area that can't be seen in a single shot with conventional 3D ultrasound scanners."
The gradient-registration/image mosaic technique is not limited to medical ultrasound. In fact, says Cohen-Or, "it can be used as a general direct method for image registration for any type of noisy images. [MedSim] is considering its use in clinical [medical ultrasound] equipment, but it can also be used for such common applications as stitching together several photographs to build a panoramic view of a scene."
Diana Phillips Mahoney is chief technology editor of Computer Graphics World.