Harvard BioRobotics Laboratory: Research Areas

Research Areas

Surgical Robotics
Tactile Sensing and Display
Tactile Imaging
Teleoperated Robotics
Tissue Mechanics
Robotic Grasping and Shape Deposition Manufacturing
Biological Motor Control

Surgical Robotics

Guiding Intracardiac Beating Heart Procedures With Real-Time Three-Dimensional Ultrasound: To perform procedures inside a patient's heart (intracardiac surgery), cardiopulmonary bypass is necessary so the surgeon can work on a relaxed open heart. Although this technique is the current standard, new studies have identified numerous adverse effects of a cardiopulmonary bypass. Minimally invasive procedures could eliminate the need for a cardiopulmonary bypass, thereby allowing the surgeon to work directly inside the beating heart. Unfortunately, there currently is no suitable imaging modality to facilitate intracardiac beating heart surgery as traditional endoscopes are ineffective due to the opacity of blood. However, recent developments by Philips Medical Systems have yielded a new real-time three-dimensional ultrasound system which may enable beating heart intracardiac surgery.
Virtual Fixtures for Robotic Cardiac Surgery: Traditional methods in cardiothoracic surgery require large incisions such as sternotomies. Recent work has demonstrated that robotic systems enable execution of coronary artery bypass graft (CABG) procedures through small incisions. In practice, these systems have proved cumbersome to use, with shortcomings that include decreased visual and haptic information, motion constraints, and the need for cognitive spatio-motor remapping from the surgeon's hands to the instrument space. We propose to help alleviate these difficulties through virtual fixtures. A virtual fixture is a computer-generated constraint that simplifies task execution by reducing precision requirements or the number of degrees of freedom that must be controlled. On a desktop computer, these fixtures are analogous to computer mouse features such as snap-to-grid that make it simpler to precisely position the cursor. In this study, the validity of the concept of virtual fixtures is tested on the ZEUS surgical robot system (Computer Motion, Inc., Goleta, Calif.).
Force Feedback in Surgery: An Analysis of Blunt Dissection Force feedback is widely assumed to enhance performance in robotic surgery, but its benefits have not yet been systematically assessed. In this study we examine the effects of force feedback on a blunt dissection task. Subjects used a telerobotic system to expose an artery in a synthetic model while viewing the operative site with a video laparoscope. Performance was compared between force feedback gains of 75% and 150% and no force feedback. The absence of force feedback increased the average force magnitude applied to the tissue by at least 50%, and increased the peak force magnitude by at least a factor of 2. The number of errors that damage tissue increased by over a factor 3. The rate and precision of dissection were not significantly enhanced with force feedback. We hypothesize that force feedback is helpful in this blunt dissection task because the artery is stiffer than the surrounding tissue. This mechanical contrast serves to constrain the subjects’ hand from commanding inappropriate motions that generate large forces.
Autonomous Localization of the Internal-Mammary Artery using a Tactile Sensor Array: In an effort to decrease the length of time in the OR for Minimalliy Invasive Coronary Artery Bypass Grafting, semi-autonomous localization of the Internal Mammary Artery (IMA) through tactile sensing is being investigated. The surgical robot holds a pressure sensing array. The surgeon initializes the system by placing the sensor on the IMA in two separate locations. The system then moves the sensor, following the IMA’s pressure variations and creating a map of the IMA’s location. That map can allow various surgical aids for the upcoming procedure, including visual guidance and the creation of virtual walls to protect the artery.

Tactile Sensing and Display

Remote Palpation Instruments for Minimally Invasive Surgery:
We are developing remote palpation systems to convey tactile information from inside a patient's body to the surgeon's fingertips during minimally invasive procedures. These new instruments will contain tactile sensors that measure pressure distribution on the instruments as tissue is manipulated. The signals from these sensors will be sampled by a dedicated computer system, which will apply appropriate signal processing algorithms. Finally, the tactile information will be conveyed to the surgeon through tactile "display" devices that recreate the remote pressure distribution on the surgeon's fingertips. Creation of remote palpation technology will increase safety and reliability in present minimally invasive procedures, and bring the advantages of minimally invasive techniques to other, more complex procedures, which are not possible today.

A Tactile Shape Display Using RC Servomotors
Tactile displays are used to convey small-scale force and shape information to the tip of the finger. In this paper, we present a 6x6 tactile shape display that uses commercial RC servomotors to actuate an array of mechanical pins. The display has a maximum pin deflection of 2 mm along with a resolution of 4 bits. Pin spacing is 2 mm with a pin diameter of 1 mm. The display can accurately represent frequencies up to 25 Hz for small amplitudes and is slew rate limited at 38 mm/sec for larger amplitudes.

Tactile Shape Displays:
The tactile display in our prototype system consists of a line of 10 individually actuated pins that are raised against the fingerpad. Shown below is a drawing of our design. A line configuration was chosen since the palpation instrument will be scanned across the tissue allowing motion to provide the other dimension. This also simplifies the design. SMA wires are used to drive the pins. As electric current heats the wire, it goes through a phase transformation and shortens, thus pushing the pin up. With this design, each pin can move 3 mm and produce over 1 N of force. A primary problem with SMA is the slow response times. We overcame this by using water cooling and position feedback for each pin from optical sensors. Figure 5 shows the response of the pin as a function of desired position frequency. The output displacement drops by 30% (-3 db point) at 40 Hz. This satisfies the design specification set by the finger speed experiments.

Vibrotactile Sensing and Display:
We have developed tactile sensors and display for measuring and conveying task-related vibrations in robotic manipulation, teleoperation and virtual environments. Vibration displays can be implemented with inexpensive, open loop devices that can be added to many existing manipulation systems to improve performance. Aside from developing vibration sensing and display devices, we have worked to delineate the kinds of tasks where high frequency vibratory feedback is important. In inspection and exploration tasks the detection of vibrations can be the fundamental goal of the task, while in some manipulation tasks vibrations can enhance performance by reducing reaction times or permitting minimization of forces. We have also developed a vibration sensing and display system for a high-capacity undersea teleoperated robot, and field tested it on an offshore oil platform.

Tactile Imaging

Tactile Imaging of Breast Tissue:
Tactile Imaging is a way of non-invasively imaging palpable features, especially in soft tissue such as that of the breast. The tactile image obtained by gliding the pressure-sensitive scanhead across the region of interest is an image of the combined effects of underlying stiffness and the geometry in which it is distributed. Current work focuses on obtaining the relationship between underlying stiff-lump parameters and the resultant tactile map.

Teleoperated Robotics

World Modeling by Tele-Manipulation:
At present, teleoperation is the only way that robots can perform sophisticated manipulation tasks in unstructured environments. In this control mode, the human operator performs all required sensing and planning, and generates all motion commands based on feedback from the remote environment. In practical teleoperation systems (e.g. undersea operations, tele-surgery, etc ), the sensory feedback is often limited to video images without force feedback, which greatly restricts dexterity and productivity. We have been working to alleviate this situation by using information from the remote robot arm's sensors to assist in teleoperated manipulation tasks . We have derived algorithms that identify essential local geometric properties of objects in the remote environment including geometry, dimensions, and orientations.

Tissue Mechanics

Determining the Mechanical Properties of Excised and Whole Organ Tissues:
The aim of this research is to realistically describe the non-linear behavior of non-load-bearing soft tissues (i.e. solid abdominal organs like the liver, kidney, and spleen) under slow deformation for the ultimate goal of providing a "ground truth" tissue atlas of material properties for use in medical simulations. In conjunction with CIMIT’s simulations group at MGH we will develop mathematical models of soft tissues under slow deformations typical of therapeutic manipulations, make experimental ex-vivo, in-vivo, and in-situ measurements of soft tissue organs, and determine the in-situ boundary conditions of these organs by combining modeling, imaging, and experimental efforts.

Picture courtesy Stephane Cotin (thesis)

Biomechanics of the Human Larynx:
The larynx is a delicately tuned instrument needed to produce over one half of the sounds in the English language and to create pitch contours in song and speech. Structural changes to the larynx may increase the effort needed to speak, decrease the quality of sound produced, cause pain, or eliminate the ability to speak. These symptoms can be extremely detrimental, especially if one's career relies on voice. Better understanding of the causes underlying structural changes to the larynx and of the connection between structural and functional changes has the potential to improve prevention and treatment of voice pathologies such as vocal nodules and scar.
The goal of this project is to address questions regarding the etiology and treatment of common voice pathologies. Finite element methods are being used to construct representations of the soft tissue vocal folds and of the airflow between them with physiologically relevant variables. The model parameters are based on laryngeal geometric and material measurements contained in the literature. A second part of the project involves the design and application of in vivo experimental measurements. These will aid in the understanding of voice biomechanics and will be used to validate the theoretical models.

Modeling Heart Tissue to Avoid Injury During Trans-atrial Surgical Procedures:
Surgical procedures inside a beating heart can be performed under image guidance; however, performing these procedures requires insertion of 2 or more surgical instruments through the wall of the upper heart chamber (atrium). Placing loads on these instruments during the surgical procedure can potentially tear the heart tissue between the instruments and around the insertion sites. In an attempt to limit these injuries, we have modeled the atrial wall with a well-described strain energy function (Smaill & Hunter, 1991) under quasi-static loads to determine the optimal arrangement for multiple trans-atrial instruments. Preliminary results show that injury to the heart wall can best be limited by placing instrument ports so that the loading occurs in the cross-fiber direction. If this is not possible, instruments should be separated by the greatest distance possible to avoid inadvertent tearing of the heart wall during these surgical procedures.

Mechanics of the Human Fingerpad:
We investigated the dynamic response of the human fingerpad in vivo to a compressive load. A flat probe indented the fingerpad at a constant velocity, then held a constant position. The resulting force (0 - 2 N) increased rapidly with indentation, then relaxed during the hold phase. A quasi-linear viscoelastic model (Fung 1993) successfully explained the experimental data. The instantaneous elastic response increased exponentially with position, and the reduced relaxation function included three decaying exponentials (with time constants of approximately 4 msec, 70 msec and 4 sec) plus a constant. The model was confirmed with data from sinusoidal displacement trajectories.

Robotic Grasping and Shape Deposition Manufacturing

Robust Robotic Mechanisms and Sensors via Shape Deposition Manufacturing
One of the greatest successes of biologically-inspired design has been the development of mechanically robust robots. One promising biomimetic facbrication technique is Shape Deposition Manufacturing (SDM), which alternates material deposition and machining to produce robot structures with compliant joints and embedded sensing and actuation elements. We explore the benefits of using Shape Deposition Manufacturing for constructing a simple two-fingered gripper and add to the tools available to robot designers by developing a range of sensing modalities compatible with the process. These include Hall-effect sensors for joint angle sensing, embedded strain gauges for 3 axis force measurements, optical reflectance sensors for tactile sensing, and piezoelectric polymers for contact detection. In addition to a simple construction process, the resulting parts are extremely robust, fully functional after high impact loads and other forces due to unintended contact.

Compliant Grasping for Unstructured Environments
Compliance conveys several advantages for robotic grasping. In unstructured environments, sensing uncertainties are large and target object size and location may be poorly known. Finger compliance allows the gripper to conform to a wide range of objects while minimizing contact forces. Robot joint compliance or stiffness has often been considered in the context of active control, where active control uses sensors and actuators to achieve a desired force-deflection relationship. In contrast, passive compliance, implemented through springs in robot joints, offers additional benefits, particularly in impacts, where control loop delays may lead to poor control of contact forces. The reduced need for the sensing required to create active compliance can also lead to lower implementation costs.

"Soft" Grippers and Bugbots:
Collaborators from Stanford and UC Berkley have designed and produced a robot modeled after the cockroach, utilizing knowledge of the insect's locomotion characteristics and new manufacturing techniques. The interesting feature of the robot is a passive rubber spring joint connecting the legs to the body. This joint, mimicking the springy, resilin lined joints of the insect, aids in disturbance rejection, accomplished without sensory feedback. Our contribution to the project will be a mechanical gripper designed using similar, passive spring joints with variable stiffness, which will aid in the task of grasping an object in a unfamiliar environment, without the use of complex sensory technology.

Biological Motor Control

A Quantitative Investigation of the Effects of Hypnosis on Stroke Recovery:
Physical rehabilitation after stroke requires complex interactions between the mind, brain and body. My research focuses on characterizing these interactions through biomechanical analysis of movement during physical therapy, medical imaging of neurological changes in the motor control system and hypnosis as a means of altering the mental state of stroke patients. By combining quantitative methods with a holistic approach to rehabilitation I hope to gain insight into how mind-body interactions affect motor recovery after stroke.

Mechanical Impedance of the Human Hand:
We have worked to determine how humans modulate the mechanical impedance of their hands in response to task requirements. The results help explain sensing and motor control strategies in dextrous manipulation. Our approach involves experimental measurement of force-motion relationships of the hand and fingers during task execution. These studies have measured the impedance of the index finger in extension and abduction, and the impedance of the precision pinch grasp during lifting. We have also examined learning of impedance strategies and applications of these results to fast tasks like drumming.

Previous Research

Harvard BioRobotics Laboratory Home

	Remote Palpation Instruments for Minimally Invasive Surgery: We are developing remote palpation systems to convey tactile information from inside a patient's body to the surgeon's fingertips during minimally invasive procedures. These new instruments will contain tactile sensors that measure pressure distribution on the instruments as tissue is manipulated. The signals from these sensors will be sampled by a dedicated computer system, which will apply appropriate signal processing algorithms. Finally, the tactile information will be conveyed to the surgeon through tactile "display" devices that recreate the remote pressure distribution on the surgeon's fingertips. Creation of remote palpation technology will increase safety and reliability in present minimally invasive procedures, and bring the advantages of minimally invasive techniques to other, more complex procedures, which are not possible today.
	A Tactile Shape Display Using RC Servomotors Tactile displays are used to convey small-scale force and shape information to the tip of the finger. In this paper, we present a 6x6 tactile shape display that uses commercial RC servomotors to actuate an array of mechanical pins. The display has a maximum pin deflection of 2 mm along with a resolution of 4 bits. Pin spacing is 2 mm with a pin diameter of 1 mm. The display can accurately represent frequencies up to 25 Hz for small amplitudes and is slew rate limited at 38 mm/sec for larger amplitudes.
	Tactile Shape Displays: The tactile display in our prototype system consists of a line of 10 individually actuated pins that are raised against the fingerpad. Shown below is a drawing of our design. A line configuration was chosen since the palpation instrument will be scanned across the tissue allowing motion to provide the other dimension. This also simplifies the design. SMA wires are used to drive the pins. As electric current heats the wire, it goes through a phase transformation and shortens, thus pushing the pin up. With this design, each pin can move 3 mm and produce over 1 N of force. A primary problem with SMA is the slow response times. We overcame this by using water cooling and position feedback for each pin from optical sensors. Figure 5 shows the response of the pin as a function of desired position frequency. The output displacement drops by 30% (-3 db point) at 40 Hz. This satisfies the design specification set by the finger speed experiments.
	Vibrotactile Sensing and Display: We have developed tactile sensors and display for measuring and conveying task-related vibrations in robotic manipulation, teleoperation and virtual environments. Vibration displays can be implemented with inexpensive, open loop devices that can be added to many existing manipulation systems to improve performance. Aside from developing vibration sensing and display devices, we have worked to delineate the kinds of tasks where high frequency vibratory feedback is important. In inspection and exploration tasks the detection of vibrations can be the fundamental goal of the task, while in some manipulation tasks vibrations can enhance performance by reducing reaction times or permitting minimization of forces. We have also developed a vibration sensing and display system for a high-capacity undersea teleoperated robot, and field tested it on an offshore oil platform.

Tissue Mechanics Determining the Mechanical Properties of Excised and Whole Organ Tissues: The aim of this research is to realistically describe the non-linear behavior of non-load-bearing soft tissues (i.e. solid abdominal organs like the liver, kidney, and spleen) under slow deformation for the ultimate goal of providing a "ground truth" tissue atlas of material properties for use in medical simulations. In conjunction with CIMIT’s simulations group at MGH we will develop mathematical models of soft tissues under slow deformations typical of therapeutic manipulations, make experimental ex-vivo, in-vivo, and in-situ measurements of soft tissue organs, and determine the in-situ boundary conditions of these organs by combining modeling, imaging, and experimental efforts.	Picture courtesy Stephane Cotin (thesis)
Biomechanics of the Human Larynx: The larynx is a delicately tuned instrument needed to produce over one half of the sounds in the English language and to create pitch contours in song and speech. Structural changes to the larynx may increase the effort needed to speak, decrease the quality of sound produced, cause pain, or eliminate the ability to speak. These symptoms can be extremely detrimental, especially if one's career relies on voice. Better understanding of the causes underlying structural changes to the larynx and of the connection between structural and functional changes has the potential to improve prevention and treatment of voice pathologies such as vocal nodules and scar. The goal of this project is to address questions regarding the etiology and treatment of common voice pathologies. Finite element methods are being used to construct representations of the soft tissue vocal folds and of the airflow between them with physiologically relevant variables. The model parameters are based on laryngeal geometric and material measurements contained in the literature. A second part of the project involves the design and application of in vivo experimental measurements. These will aid in the understanding of voice biomechanics and will be used to validate the theoretical models.
Modeling Heart Tissue to Avoid Injury During Trans-atrial Surgical Procedures: Surgical procedures inside a beating heart can be performed under image guidance; however, performing these procedures requires insertion of 2 or more surgical instruments through the wall of the upper heart chamber (atrium). Placing loads on these instruments during the surgical procedure can potentially tear the heart tissue between the instruments and around the insertion sites. In an attempt to limit these injuries, we have modeled the atrial wall with a well-described strain energy function (Smaill & Hunter, 1991) under quasi-static loads to determine the optimal arrangement for multiple trans-atrial instruments. Preliminary results show that injury to the heart wall can best be limited by placing instrument ports so that the loading occurs in the cross-fiber direction. If this is not possible, instruments should be separated by the greatest distance possible to avoid inadvertent tearing of the heart wall during these surgical procedures.
Mechanics of the Human Fingerpad: We investigated the dynamic response of the human fingerpad in vivo to a compressive load. A flat probe indented the fingerpad at a constant velocity, then held a constant position. The resulting force (0 - 2 N) increased rapidly with indentation, then relaxed during the hold phase. A quasi-linear viscoelastic model (Fung 1993) successfully explained the experimental data. The instantaneous elastic response increased exponentially with position, and the reduced relaxation function included three decaying exponentials (with time constants of approximately 4 msec, 70 msec and 4 sec) plus a constant. The model was confirmed with data from sinusoidal displacement trajectories.

	Robust Robotic Mechanisms and Sensors via Shape Deposition Manufacturing One of the greatest successes of biologically-inspired design has been the development of mechanically robust robots. One promising biomimetic facbrication technique is Shape Deposition Manufacturing (SDM), which alternates material deposition and machining to produce robot structures with compliant joints and embedded sensing and actuation elements. We explore the benefits of using Shape Deposition Manufacturing for constructing a simple two-fingered gripper and add to the tools available to robot designers by developing a range of sensing modalities compatible with the process. These include Hall-effect sensors for joint angle sensing, embedded strain gauges for 3 axis force measurements, optical reflectance sensors for tactile sensing, and piezoelectric polymers for contact detection. In addition to a simple construction process, the resulting parts are extremely robust, fully functional after high impact loads and other forces due to unintended contact.
	Compliant Grasping for Unstructured Environments Compliance conveys several advantages for robotic grasping. In unstructured environments, sensing uncertainties are large and target object size and location may be poorly known. Finger compliance allows the gripper to conform to a wide range of objects while minimizing contact forces. Robot joint compliance or stiffness has often been considered in the context of active control, where active control uses sensors and actuators to achieve a desired force-deflection relationship. In contrast, passive compliance, implemented through springs in robot joints, offers additional benefits, particularly in impacts, where control loop delays may lead to poor control of contact forces. The reduced need for the sensing required to create active compliance can also lead to lower implementation costs.
	"Soft" Grippers and Bugbots: Collaborators from Stanford and UC Berkley have designed and produced a robot modeled after the cockroach, utilizing knowledge of the insect's locomotion characteristics and new manufacturing techniques. The interesting feature of the robot is a passive rubber spring joint connecting the legs to the body. This joint, mimicking the springy, resilin lined joints of the insect, aids in disturbance rejection, accomplished without sensory feedback. Our contribution to the project will be a mechanical gripper designed using similar, passive spring joints with variable stiffness, which will aid in the task of grasping an object in a unfamiliar environment, without the use of complex sensory technology.

Biological Motor Control
A Quantitative Investigation of the Effects of Hypnosis on Stroke Recovery: Physical rehabilitation after stroke requires complex interactions between the mind, brain and body. My research focuses on characterizing these interactions through biomechanical analysis of movement during physical therapy, medical imaging of neurological changes in the motor control system and hypnosis as a means of altering the mental state of stroke patients. By combining quantitative methods with a holistic approach to rehabilitation I hope to gain insight into how mind-body interactions affect motor recovery after stroke.
Mechanical Impedance of the Human Hand: We have worked to determine how humans modulate the mechanical impedance of their hands in response to task requirements. The results help explain sensing and motor control strategies in dextrous manipulation. Our approach involves experimental measurement of force-motion relationships of the hand and fingers during task execution. These studies have measured the impedance of the index finger in extension and abduction, and the impedance of the precision pinch grasp during lifting. We have also examined learning of impedance strategies and applications of these results to fast tasks like drumming.