Real-time 3D Ultrasound Mosaicing and Visualization

Cardiac 3D ultrasound imaging has enabled new minimally invasive heart procedures. 3DUS is non-invasive, inexpensive, able to image through the blood, and can provide real-time diagnostic information of the anatomical region of interest. 3DUS also mitigates the difficulties in spatial perception associated with traditional 2DUS.

Its narrow field of view, however, limits its efficacy in guiding beating heart procedures where geometrically complex and spatially extended moving anatomic structures are often involved. We developed a system that performs electrocardiograph (ECG) gated 4D mosaicing and visualization of 3DUS volumes. Real-time operation is enabled by GPU implementation. Brattain and Howe, MICCAI 2011


Slice View shows instrument tip in noisy 3DUS volume and provides a clear view of instrument tip & tissue interaction.

  • Cut planes automatically generated from EM sensor’s axes
  • Adjust the angle and thickness of the cut planes on the fly
  • Highlight the instrument tip for better tip & tissue contact tracking
Brattain and Howe, IPCAI 2012

Intracardiac Ultrasound Imaging and Image Processing

We have developed a system for steering cardiac imaging catheters to automatically visualize heart structures and instruments. The goal of this system is to improve the efficacy, speed, and safety of imaging during catheter-based arrhythmia treatment procedures. Many cardiac arrhythmias can be effectively treated by radiofrequency (RF) catheter ablation, but the primary imaging modality during an ablation procedure is x-ray fluoroscopy, which does not effectively distinguish soft tissues and exposes patients and clinical staff to ionizing radiation.

Real time visualization of intracardiac structures is needed to further improve the efficacy of catheter ablation procedures. Intracardiac echocardiography (ICE) catheters are routinely used during ablation procedures for their high-quality ultrasound soft tissue visualization. A major challenge during catheter ablation is the lack of acute lesion assessment, and ICE visualization of lesion growth has the potential to improve acute procedural outcomes. However, manual use of ICE requires specialized training and maintaining continued alignment of the imaging plane is a significant challenge. Existing robotic catheter systems (e.g. Hansen Medical,, Stereotaxis, enable teleoperation of catheter controls and reduce radiation exposure, but they do not solve the problem of steering an ICE imaging plane towards a desired structure.

In this study our system manipulates an ICE catheter in a water tank to visualize a phantom. Multiple images are stitched together to create a volume of the target structure. This enablement of real time visualization during catheter ablation has potential to facilitate improved long-term treatment of cardiac arrhythmias. Brattain, Loschak, Tschabrunn, Anter and Howe, Hamlyn Symposium 2013

Fig. 1. ICE catheter actuated degrees of freedom with corresponding tip and imaging plane motions
Fig. 2. (a) Schematic of imaging plane rotation while visualizing CAD model of phantom, (b) example handle inputs for large image plane rotation, (c) data for 23° total imaging plane rotation
Fig. 3. Ultrasound image slices stitched into a volume


Laura J. Brattain, Paul M. Loschak, Cory M. Tschabrunn, Elad Anter, Robert D. Howe
Robotic Steering of Cardiac Ultrasound Imaging Catheters
Proceedings of the 6th Hamlyn Symposium on Medical Robotics, London, UK, June 22-25, 2013.

Paul M. Loschak, Laura J. Brattain, Robert D. Howe
Automated Pointing of Cardiac Imaging Catheters
IEEE Int. Conf. on Robotics and Automation, Karlsruhe, Germany, 2013

Laura J. Brattain and Robert D. Howe
Enabling 3D Ultrasound Procedure Guidance through Enhanced Visualization
P. Abolmaesumi, L. Joskowicz, N. Navab and P. Jannin (Eds.): IPCAI 2012, LNCS 7330, pp. 115-124, 2012. Springer, Heidelberg (2012)

Laura J. Brattain and Robert D. Howe
Real-time 4D Ultrasound Mosaicing and Visualization
G. Fichtinger, A. Martel, and T. Peters (Eds.): MICCAI 2011, Part I, LNCS 6891, pp. 105-112, 2011. Springer, Heidelberg (2011)