The project Visual Enabling for Precision Surgery (VEP) will develop a computer-based three-dimensional (3D) image guidance system for neurosurgical interventions which are supported by an interventional magnetic resonance imaging (MRI) system from General Electric. Interventional MRI systems can generate intraoperative MR images which give important feedback for the precise positioning of surgical devices. One specific intervention technique considered within this project is the laser-induced interstitial thermotherapy (LITT) where an optical fiber is implanted within a brain tumor to be coagulated by laser energy.
Technically the system will be realised as an augmented reality system combining various types of medical and synthetic images. The physician will experience the surgical orientation environment as a kind of interactive graphical 3D scene showing all relevant data for the intended intervention. The design approach requires close interdisciplinary co-operation among computer scientists, medical physicists, and human factors specialists with radiologists, surgeons, and other medical experts. The common goal is to improve the spatial and structural orientation and thereby the quality and precision of neurosurgical interventions.
The VEP system is an example of a medical enabling system which can be attached to new medical high-tech systems in order to make them intuitively accessible for the average physician. The approach tackles the problem field effective enabling mainly with orientation devices which can be combined with existing or new devices and thus used on the job. Additionally, related training systems in form of interactive CD-ROMs will be developed.
The project started in May 1997. A first result was the LOCALITE guidance system for minimally invasive neurosurgery which enhances the functionality of the iMRI system SIGNA SP from GE Medical Systems equipped with a FlashPoint 5000 locator device from Image Guided Technologies, Inc. The LOCALITE guidance system got the CE-certification in May 2000 and is now in routine application for brain surgery at the Klinikum Krefeld, Germany.
Neurosurgical interventions rely heavily on imaging techniques like computed tomography (CT), magnetic resonance imaging (MRI), positron-emission tomography (PET), ultrasound, etc. The medical tasks include preoperative diagnosis and planning of the intervention as well as operative therapy and control based on available imaging sources. The surgeon needs good spatial and structural orientation to perform these tasks which is not readily provided by an unrelated set of images of different quality and content.
The focal element of the VEP system is a 3D reference scene which integrates registered (i.e., geometrically aligned) 3D and 2D data sets of various available imaging sources to provide an orientation environment (visual enabling system) which supports the surgeon in planning and performing neurosurgical interventions.
In the preoperative phase, that method of intervention is selected which is best suited for this patient. For neurosurgical interventions, the least harmful tumor access is chosen which avoids critical brain structures.
For the laser-induced interstitial thermotherapy, exposition times and distribution parameters have to be chosen such that most of the tumor is coagulated without affecting neighbouring brain tissue.
The intervention requires a precise placing of a catheter, an optical fiber tip or another surgical device within the tumor. The access to the tumor is chosen such that essential brain areas and blood-vessels are circumvented. Intraoperatively, the placement can be controlled by 2D images produced by the interventional MRI system during the intervention. The images have to be registered with the preoperative reference scene together with synthetic images of access path and surgical device to provide the surgeon with the necessary spatial and structural orientations.
For the laser-induced interstitial thermotherapy, the tumor tissue is coagulated by laser energy as a second step. The propagation of the coagulation process depends on various parameters (e.g., laser energy, absorption, blood perfusion) and has to be controlled whether the coagulation follows the anticipated pattern. Otherwise, the exposition time and dose have to be adapted. Temperature distribution can be controlled by images produced by the interventional MRI system during the intervention.
The various imaging modalities available for radiologists and surgeons offer new opportunities for a deeper insight into the pathological situation within the human body. The move from two-dimensional (2D) images to three-dimensional (3D) data sets improves the understanding of the geometrical situation (e.g., position of a tumor within the surrounding tissues) and supports the planning and performance of an intervention. Combined with precise registration techniques, instruments can be guided during the intervention and precisely be positioned. The wealth of images can only be exploited when different modalities are fused and presented to the physician in an intuitive and easily understandable way.
For building up an integrated orientation concept, the image planes measured by MR or CT systems have to be combined into one 3D data set which can be visualised such that the position of a tumor can be easily identified and correlated with neighbouring areas of the brain. Using appropriate projections or stereo viewing techniques, the physician can immediately localise all relevant parts of the brain and establish an optimal access way without being referred to his complex spatial imaginative faculty.
Brain areas which support important human capabilities can be identified in the surrounding of the tumor with various imaging techniques (fMR, PET, SPECT, MEG). Angiography techniques deliver 3D data of the vascular structures within the brain. The different modalities have to be registered with the 3D data set of the brain (MR or CT) and fused to an integrated representation which includes all relevant information about the tumor. Registration requires some kind of land marks or well defined structures in both modalities to fit one 3D data set precisely onto the other.
The data taken from the patient can be enhanced by synthetic structures showing the intended access path and the instruments such that the intended intervention can be performed and tested experimentally. For the laser-induced interstitial thermotherapy (LITT), the enhanced 3D data set can be used for simulating the thermotherapy and visualising its effect.
|
|
|
|
|
|
|
|
Unfortunately, the preoperative data set is not sufficient for precisely localising a tumor within the brain. The position of a tumor and the form of its surrounding soft tissues may change when surgical instruments are inserted due to the soft nature of the brain and the surrounding liquid. The changes may be considerably such devaluating the preoperative data set.
Intervention microscopes deliver up-to-date images from the intervention channel and the tissue reached. The position of the microscope can be measured which allows the current image of the microscope to be compared with the expected image from the reference scene. If identical landmarks or structures can be identified in microscope image and reference scene, the reference scene can be locally morphed to reflect the current geometry of the brain.
 |
Interventional MRI |
Interventional MR systems allow for a continuous monitoring of the brain typically in a plane through the tip of the intervention device. Whereas a microscope can see only the immediate neighbourhood, interventional MR can look ahead and to the sides which gives more information and security for planning the next step. As the quality and the number of slices generated by an interventional MR is worse than with the preoperative MR, there is a need for registrating and fusing the current slices of the interventional MR with the high quality reference scene.
 |
upper: overview of scull with interventional device and associated scan plane; left: plane from interventional MRI scan (low quality); right: plane from preoperative MRI scan (high quality). Quicktime movie (180 KB) showing positioning of device. |
MR systems can also be used to measure the temperature within the brain during a laser-induced interstitial thermo therapy. As the coagulation process is difficult to predict due to irregularities in tissues and thermo convection, an interventional MR system can be used to control whether the therapy proceeds as planned or whether the plan has to be modified. This asks for a fusion of current MR images with respective images of the therapy planning.
Ultrasound is a further modality considered within the project. Ultrasound can be used as real-time intraoperative feedback. However, low image resolution and image distortions make it difficult to integrate it with the other images.
The variety of images taken from a patient can hinder the physician if they are not offered in a well structured and target-oriented manner. Head and hand of the surgeon are fully occupied by the intervention and are not free to control a computer system. This implies that images have to be presented just in time with just the information needed by the surgeon for the next step during the intervention. He has only to compare the information provided by the computer with the current situation to see whether the intervention proceeds as planned and what are the next critical points.
This target requires the design of a cognitive adequate system based on in depth analyses of typical orientation demands and notorious problem constellations. Based on the analysis, prototype system will be implemented and tested in situ whether they provide the right information at the right time and give the surgeon just the needed information for the next decision. This does not mean to integrate all information but to select careful the task critical information and present them in semi-transparent 3D scenes enhanced with synthetic material.
This selection, integration, and presentation of 3D and 2D images is the central design task of the project. The surgeon has to understand intuitively at what point he is looking and what the images are showing. To select among different information set ups and perspectives, interaction techniques like voice input have to be integrated to keep the hands free for the surgical task. These Human Computer Interaction (HCI) problems are the focus of the project.
 |
left: texture-based rendering of a 3D-CT-volume in real-time right: shadows added to improve depth perception Quicktime movie (610 KB) |
Sven Arnold, Uwe Behrens, Martin Bublat, Harald Busse, Manja Fieberg, Gernoth Grunst, Marko Jahnke, Matthias Jungmann, Klaus Kansy, Ralf Ratering, Arno Schmitgen, Ron Schwarz, Hans-Joachim Schwarzmaier, Peter Wisskirchen, Marc Wengler
Please contact: Klaus Kansy, FhG-FIT