Last updated: 2/12 at 9:00P.M.

The goal of my project is automate the placement and control of ultrasound probes during minimally invasive spinal procedures with the goal of lowering cost to the patient and radiation exposure to the surgeon.

• Students: Joshua Shubert
• Mentor(s): Muyinatu Bell

Due to advances in medicine, many procedures can now be performed in a minimally invasive manner. However for some Spinal procedures such as spinal fusions, kyphoplasty and vertebroplasty this means the use of X-ray imaging to guide the procedures.

The aim of this project is to simultaneously reduce radiation exposure to patient and surgeon and also lower the cost of the procedure by using inter-operative photoacoustic and ultrasound imaging.

• Minimum: (Expected by March 20th)
  1. Robot Control Software Package with
     1. Human Shape Segmentation
     2. Inverse Kinematics for Probe Placement
     3. Contact Force Feedback
     4. A Nice GUI
  2. Demonstration of Autonomous Probe Placement (video)

• Expected: (Expected by April 10th)
  1. Exploration of Photoacoustic Imaging in the Spine
     1. Images of fiber-fed needle inside vertebrae
     2. Images of bare fiber inside vertebrae (if possible)
     3. Report detailing the relationship between laser energy, depth inside vertebrae, and the resulting Photoacoustic Images
  2. Updated Needle Segmentation algorithm if necessary

• Maximum: (Expected by May 15th)
  1. Expanded Robot Control Software Package with
     1. Needle Tip Segmentation
     2. Visual Servoing to Track the Needle Tip
     3. Visual Display of Photoacoustic signal overlaid onto B-Mode Ultrasound
  2. Demonstration of entire system on either a human-shaped spine phantom or a cadaver

My project is split into 3 main parts:

1. Preoperative Autonomous Placement of Ultrasound Probe onto Spinal Region
   1. Calibration of Kinect v2 Intrinsic Parameters
   2. Registration of Kinect v2 Coordinate Frame to Sawyer Robot base Coordinate Frame
   3. Inverse Kinematics to Navigate Ultrasound Probe from an initial position to the patient's spine
   4. Force Feedback is turned on once the Ultrasound Probe begins to descend toward the patient
2. “Off-Line” Exploration into the Viability of Photoacoustic Imaging from inside a needle inside a Vertebra
   1. Spinal samples are obtained
   2. Holes are drilled or poked into the spinal samples
   3. Hollow bore needle is placed inside the spinal sample
   4. Optical Fiber is then placed inside the needle
   5. Ultrasound probe is placed so that it has good coupling to the spinal sample
   6. Laser is turned on
   7. If it is possible to get any PA signal, it will display on the Alpinion Scanner running a special PA beamforming script
   8. The Laser Energy, Needle Placement, and Probe Placement are all tweaked to explore the viability of using PA imaging of the spine intraoperatively
3. Intraoperative Autonomous Tracking using Visual Servoing of a PA Imaged needle inside a Vertebra
   1. A Needle Segmentation algorithm is developed to determine the pixel coordinates of the needle tip in the PA image in real time
   2. Ultrasound Probe calibration is performed to be able to map the pixel coordinates of the needle tip into 3-D coordinates in the Robot Base Coordinate Frame
   3. This can be used to keep the probe centered over the needle tip by moving the probe till the pixel difference between the needle tip and the center of the PA image is brought close to zero

Dependencies:

• Sawyer Robot (Obtained)
• Opotek Laser (Obtained)
• Alpinion E-CUBE Ultrasound Scanner (Obtained)
• Hollow Bore Needle (Obtained)
• 1mm Optical Fiber (Obtained)
• Linear Array Ultrasound Probe (Obtained)
• Kinect v2 (Obtained)
• Spine Samples (Obtained)
• IRB for Cadaver Study (Optional) (Not Met)

If approval for cadaver study not obtained, I will have to create a human shaped spinal phantom. My plan for this is to remove the torso of a standard plastic mannequin and use it as a mold for making a phantom. Once the phantom is made (with a spine inside), I will reinsert it into the mannequin.

1. Milestone name: Calibrate and Register Kinect
   • Planned Date: Feb 20
   • Expected Date: Feb 27
   • Status: Completed
2. Milestone name: Human Outline Segmentation
   • Planned Date: Feb 27
   • Expected Date: Mar 13
   • Status: Completed
3. Milestone name: Explore PA in Spine
   • Planned Date: Feb 20
   • Expected Date: Mar 13
   • Status: Completed
4. Milestone name: Inverse Kinematics and Probe Placement
   • Planned Date: Mar 20
   • Expected Date: Mar 27
   • Status: Completed
5. Milestone name: Visual Servoing
   • Planned Date: Mar 27
   • Expected Date: Apr 24
   • Status: In Progress
6. Milestone name: Demonstrate Entire System
   • Planned Date: Apr 24
   • Expected Date: May 15
   • Status: In Progress

• Project Plan
  • autonomous_placement_of_ultrasound_probe_for_spinal_surgeries2.pdf
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • acis_seminar_writeup.docx
• Project Final Presentation
  • finalposter2.pptx

* here list references and reading material

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space.