This page is a work in progress. It will be updated regularly throughout the upcoming weeks.

Last updated: 2011-05-19 3:45am

The goal of this project is to design, develop, and test a borescope integrated with the steady-hand robot. This integration will allow for pre-operative safe path planning, as needed for implant surgery, during surgery and into small orifices. There will be two components of this project: a hardware adapter which integrates the borescope with the steady-hand robot, and a software component which will allow the probe to interface with current software.

• Students: Xingchi He, Saumya Gurbani, Alperen Degirmenci
• Mentor(s): R. Taylor, I. Iordachita
• Other Collaborators: W. Chien (clinical advisor - JHMI)

Cochlear implant surgery is performed on patients who suffer severe hearing loss, allowing them to perceive a broader and finer tuned range of sound. The implant has two components: an external microphone to capture sound, and an embedded electrode to transmit the sound (via electric current) to the inner ear.

One of the greatest risks involved with embedding this electrode is the possibility of damaging critical tissue, such as the facial nerve, which are located in very close proximity to the target location of the implant. Damage to the facial nerve, for example, can lead to facial paralysis. Furthermore, the diameter of the scala tympani is under 1mm, thus allowing almost no room for error during the placement of the implant.

Current procedure involved passing a flexible electrode through the round window and through the first bend of the scala tympani for placement. Pre-operative CT and MRI imaging are used to create a general view of the patient's cochlea. However, these do not give the surgeon with an accurate, safe path to follow during surgery.

Using a borescope, we will descend into the cochlear canal and image it to determine the safest path to follow when inserting another instrument. Because the borescope is attached to the same tool arm as the implant tools, a simple registration can be done between the borescope's path and the tooltip's path even though the two items are not being inserted concurrently.

We will develop the hardware necessary to attach the borescope system with the Eye Robot arm, and the software necessary to build and register a safe path.

Summary of Specific Goals:

1. Build hardware adapter to attach borescope system with Eye Robot arm
2. Develop software to create a safe path using the borescope
3. Develop software to register another tool with the safe path

• Minimum: (Expected by 05/03/2011)
  1. Build a prototype of a hardware adapter for attaching borescope to the Eye Robot
  2. Develop software to build a safe path using data collected from borescope manually

• Expected: (Expected by 05/17/2011)
  1. Create a registration function to align another surgical tool with the safe path

• Maximum: (Expected by 08/01/2011, if summer plans work out)
  1. Improve software to automatically determine safe path
  2. Integrate image processing methods to do a 3D reconstruction (optical flow) of cochlear canal
  3. Use 3D reconstruction and safe path to generate virtual fixtures for surgical tool

EyeRobot adapter mount holding the borescope fiber. The mount allows for interchangeable tooltips, so the borescope can be easily replaced with an electrode implant or any other surgical tool. This allows for fast and accurate registration of tools to imaging data.

Screenshot of our Safepath Recording GUI, which has the ability to detect and export to file a set of safepath points.

Image from borescope (left) along with segmentation (right). Both streams can be processed and displayed in real time.

Safepath generated in MATLAB from the data collected via our program. The cylinder represents the maximum trajectory error, and has a radius of only 0.57mm.

1. Milestone name: Building adapter for attaching the light source to borescope.
   • Planned Date: 04-08-2011
   • Expected Date: 04-08-2011
   • Status: NO LONGER NEEDED. We damaged the boroscope's illumination fiber. We have attached an external, single fiber for illumination.
2. Milestone name: Building first prototype adapter for mounting the borescope to EyeRobot.
   • Planned Date: 04-18-2011
   • Expected Date: 04-20-2011
   • Status: Completed.
3. Milestone name: Developing software for manually creating safest insertion path.
   • Planned Date: 04-25-2011
   • Expected Date: 04-22-2011
   • Status: Completed; requires validation.
4. Milestone name: Building second prototype adapter for mounting the borescope to EyeRobot.
   • Planned Date: 05-03-2011
   • Expected Date: 05-12-2011
   • Status: Completed. An additional adapter for the cochlear implant was also built.
5. Milestone name: Image segmentation software.
   • Planned Date: 05-16-2011
   • Expected Date: 05-16-2011
   • Status: Completed.
6. Milestone name: Software to register borescope to other surgical tools with the safe path.
   • Planned Date: 05-17-2011
   • Expected Date: Summer 2011
   • Status: Not started, will not be completed this semester.

• We changed the imaging system from OCT to micro-borescope
• We damaged the light cable port for the borescope; now we cannot get light through the scope. We have attached an external, single fiber to illuminate the cochlear canal.

• Project Plan
  • Original OCT Project plan presentation
  • Orignial OCT Project plan proposal
  • Borescope Project plan proposal (NEW)
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • (Confidential)
• Paper Seminar Presentations
  • Alperen
    • A Stable Optic-Flow Based Method for Tracking Colonoscopy Images
    • Critical Review
    • Seminar Presentation
  • Saumya
    • A Segmentation Algorithm for a Robotic Micro-Endoscope for Exploration of the Spinal Cord (Ascari, 2004)
    • Critical Review
    • Seminar Presentation
  • Xingchi
    • Optical coherence tomography as an orientation guide in cochlear implant surgery?
    • New strategies for high precision surgery of the temporal bone using a robotic approach for cochlear implantation
    • Critical Review
    • Seminar Presentation
• Project Final Presentation
  • PDF of Poster
• Project Final Report
  • Final Report
  • Safe Path GUI Documentation
  • Borescope Viewer/Segmentation Documentation

1. “Cochlear Implants”. Medical Devices - US Food and Drug Administration. http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/ImplantsandProsthetics/CochlearImplants/default.htm. Accessed 12 February 2011.
2. I. Fleming, M. Balicki, J. Koo, I. Iordachita, B. Mitchell, J. Handa, G. Hager, and R. Taylor, “Cooperative robot assistant for retinal microsurgery.,” Medical image computing and computer-assisted intervention : MICCAI … International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 11, Jan. 2008, pp. 543-50. pdf
3. T. Klenzner, C.C. Ngan, F.B. Knapp, H. Knoop, J. Kromeier, A. Aschendorff, E. Papastathopoulos, J. Raczkowsky, H. Wörn, and J. Schipper, “New strategies for high precision surgery of the temporal bone using a robotic approach for cochlear implantation,” European Archives of OtoRhinoLaryngology, vol. 266, 2009, pp. 955-960.pdf
4. J. Liu, K. Subramanian, T. Yoo, R.Van Uitert, “A Stable Optic-Flow Based Method for Tracking Colonoscopy Images,” CVPRW, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp.1–8, June 2008. pdf
5. Cohen, Noel L. and J. Thomas Roland Jr. “Complications of Cochlear Implant Surgery”.Cochlear Implants. Ed. Susan B. Waltzman and J. Thomas Roland. Thieme Medical Pub, 2006. 126-132.
6. K. Subramanian, T. Yoo, and R. Van Uitert, “A stable optic-flow based method for tracking colonoscopy images,” 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Jun. 2008, pp. 1-8.pdf
7. C.G. Wright, P.S. Roland, and J. Kuzma, “Advanced bionics thin lateral and Helix II electrodes: a temporal bone study.,” The Laryngoscope, vol. 115, Nov. 2005, pp. 2041-5.pdf
8. D. Zhang et. al., “Inspecting the cochlear scala tympanic with flexible and semi-flexible micro-endoscope.” Journal of Clinical Otorhinolaryngology Head and Neck Surgery, vol. 20 issue 4, 2006. pp 169-71. pdf

Source code will be committed to the CISST codebase.

The goal of this project is to design, develop, and test an OCT probe integrated with the steady-hand robot. This integration will allow for improved imaging of the cochlea, as needed for implant surgery. There will be two components of this project: a hardware adapter which allows an OCT probe (or multiple probes) to attach to the steady-hand robot, and a software component which will allow the probe to interface with current software.

• Students: Xingchi He, Saumya Gurbani, Alperen Degirmenci
• Mentor(s): R. Taylor, I. Iordachita
• Other Collaborators: W. Chien (clinical advisor - JHMI)

Cochlear implant surgery is performed on patients who suffer severe hearing loss, allowing them to perceive a broader and finer tuned range of sound. The implant has two components: an external microphone to capture sound, and an embedded electrode to transmit the sound (via electric current) to the inner ear.

One of the greatest risks involved with embedding this electrode is the possibility of damaging critical tissue, such as the facial nerve, which are located in very close proximity to the target location of the implant. Damage to the facial nerve, for example, can lead to facial paralysis. Furthermore, the diameter of the scala tympani is under 1mm, thus allowing almost no room for error during the placement of the implant.

Current procedure involved passing a flexible electrode through the round window and through the first bend of the scala tympani for placement. Pre-operative CT and MRI imaging are used in order to properly plan the surgery. While both of these provide a layout of the cochlea and give the surgeon a path to follow, they do not mitigate the risk of harm during the actual placement.

The integration of an optical coherence tomography (OCT) probe into the electrode placement probe would allow for real time imaging during the surgery. The surgeon would be able to visualize the surrounding of the probe’s tip and ensure that the probe will avoid striking any structural components during placement. Attaching such probes to a steady-hand robot will further mitigate the risk of damage due to hand tremor.

Currently, the hardware and software for the steady-hand robot allow five degrees of motion: translation, plus rotation about the base of the tool-attachment. The sixth degree of motion, rotation about the axis of the tool, is currently not available. Since this rotation is necessary to produce 360-degree imaging capabilities for the OCT probe, it must be developed.

Default Text:

Here give a short 1-2 paragraph description of the problem you are solving. Explain why this is an important problem, to whom it is important, and why. Explain briefly the project's relationship to relevant prior art and other background, giving special attention to the technical context or ongoing work that you are using for your project. I.e., many of the CIS projects are part of, or rely on, ongoing research efforts here at JHU, or they use infrastructure from existing labs. Provide a short numbered list of specific aims. Something along the lines of the following.

Our specific aims are:

1. specific aim 1
2. specific aim 2
3. specific aim 3

• Minimum: (Stages 1-2)
  1. Develop a (rotationally-free) adaptor for OCT probe to be attached to the steady-hand robot
  2. Develop a mechanism for the rotation of the OCT probe inside the cochlear canal
  3. Develop software to control the rotation of OCT probe inside the cochlear canal

• Expected: (Stage 3)
  1. Integrate a second imaging fiber into the OCT probe, directed forward, to increase maneuverability and the field-of-view
  2. 3D reconstruction of the cochlear canal using the software available

• Maximum: (Stages 4-5)
  1. Develop a hardware adapter which holds the electrode of the implant for insertion
  2. Design a bendable OCT probe
  3. Generate virtual fixture from the 3D reconstruction of the cochlear canal
  4. Suggest safe insertion paths to the surgeon
  5. Provide proximity-scaled force-feedback to the surgeon

Our project is broken up into essentially 5 stages: First OCT Probe Design, Testing, Second OCT Probe Design, Surgical Safety Adaptations, and Bendable OCT Probe Design. The probe design stages (Stages 1 and 3) are further broken down into two sub-stages, hardware and software.

Stage 1H (hardware): In this stage, our goal is to successfully build a prototype of a side-view OCT probe adapter. We will first sketch various plausible designs for the adapter and determine, with consulting from our mentors and Dr. Chien, which is the most feasible and convenient for a surgeon to use. We will then produce a CAD model of the desired design, and send it in to the machine shop for fabrication.

Stage 1S (software): Concurrent with the hardware stage, we will also begin to work on a software package to interface our adapter with the current steady-hand robot software.

Stage 2: The second stage will consist of testing our initial prototype and software on a phantom cochlear bone. This stage is critical as it will elucidate shortcomings in our design and software implementation. Iterative changes will be made, and a second prototype will be fabricated.

Stage 3H (hardware): Once we have a side-view OCT probe adapter built, we will work on modifying the adapter to incorporate a second, front-facing OCT probe. By using both these probes, we should be able to get a complete hemispherical view of the inner ear as the probes traverse it.

Stage 3S (software): Concurrently, we will write software to help us merge together the outputs from each probe to create a 3D reconstruction of the cochlea.

Stage 4: The goal of stage four is to develop software to further aid the surgeon during the implant operation. Based off of pre-operative MRI and CT imaging, we will develop virtual fixtures that our OCT probes will be able to “see” during the surgery. We will also use this data to create safe insertion paths for the surgeon to follow. Finally, we will incorporate proximity-scaled force feedback to keep the surgeon on the delineated paths.

Stage 5: This stage is our maximal possible deliverable, with the goal to create a new OCT probe itself (not just an adapter) which can be bent around the inner ear canals and still allow enough light to pass so that the OCT probe can function.

Default Text:

Give a short paragraph describing your technical approach to solving the problem. In doing this, you may well want to create and refer to new pages with more detail and link to them with a suitable link. If you want to have a private link (not visible to outsiders), then see Prof Taylor. The basic link syntax for such a link would be description of private link. But we will need to set up some permissions and protections. Check with your mentor to see if this is needed.

Note that you may update this from time to time. Also, note that the more extensive discussions will be in the various reports.

A high resolution PDF of our timeline can be viewed here: timeline.pdf

We have set up the TD-OCT system in Robotorium and wrote a LabView program to interface with the TD-OCT system via a NI DAQ device. Now we can receive A-scan from the OCT and display it on the computer.

The CAD model for the probe holder and rotation mechanism has been finalized. The probe holder has been manufactured using a rapid prototyping machine. A second version has been designed and built.

A low cost, high resolution stepper motor has been ordered. It will be used for initial test of the hardware concepts and software algorithm. We need to write some code to control the motor.

We have received side-view OCT probes, however they do not perform well.

1. Milestone name: Completion of first hardware prototype adapter with rotation motor.
   • Planned Date: 03-18-2011
   • Expected Date: 03-27-2011
   • Status: In Progress. Adapter is designed in CAD and has been manufactured using a rapid prototyping machine. A DC Servo motor has been spec-ed and needs to be ordered. A low-cost stepper motor has been ordered. It will be used for initial test.
2. Milestone name: Software to analyze OCT output
   • Planned Date: 03-11-2011
   • Expected Date: 03-31-2011
   • Status: In Progress. We have a basic LabView program ready, but need to fix some minor GUI problems. We have calibrated the front-view OCT system.
3. Milestone name: Software to synchronize OCT output with Eye Robot arm motion
   • Planned Date: 03-18-2011
   • Expected Date: 04-01-2011
   • Status: Not Yet Started. We just received the OCT system, and have yet to begin this software programming.
4. Milestone name: Do a 3D reconstruction of the cochlea.
   • Planned Date: 04-17-2011
   • Expected Date: 04-17-2011
   • Status: Not Yet Started. We just received the OCT system.

1. Milestone name: Sentence or phrase explaining what it is. Include link(s) to additional pages or media as needed
   • Planned Date: xxxxxx
   • Expected Date: xxxxx
   • Status: xxxxx

• Side-view OCT Probes (Dependency): We have received three side-view probes, however it seems like they are not working as well as we would like them to.
• OCT System (Dependency): We received the system the week of March 13th, two weeks later than initially planned for. We are still waiting for Dr. Kang to provide us a working side-view OCT probe. We are now working to catch up with our original plan. We are also considering alternative solutions (FBG + front-view OCT) that could provide preliminary results sooner.
• Eye Robot 1 (Dependency): We were originally going to work on Eye Robot 2; however, we switched to Eye Robot 1 for convenience since it is less used.

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • (Confidential)
• Project Final Presentation
  • media link(s) to pdf or ppt for final poster presentation
  • etc.
• Project Final Report
  • media link(s) to pdf for project final report

NOTE: In uploading media, upload them to your own name space or to private sub-namespace, depending on whether the general public should be able to see them.

1. “Cochlear Implants”. Medical Devices - US Food and Drug Administration. http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/ImplantsandProsthetics/CochlearImplants/default.htm. Accessed 12 February 2011.
2. Cohen, Noel L. and J. Thomas Roland Jr. “Complications of Cochlear Implant Surgery”.Cochlear Implans. Ed. Susan B. Waltzman and J. Thomas Roland. Thieme Medical Pub, 2006. 126-132.
3. H.W. Pau, E. Lankenau, T. Just, and G. Hüttmann, “Imaging of Cochlear Structures by Optical Coherence Tomography (OCT). Temporal bone experiments for an OCT-guided cochleostomy technique.,” Laryngo- rhino- otologie, vol. 87, Sep. 2008, pp. 641-6. pdf
4. H.W. Pau, E. Lankenau, T. Just, D. Behrend, and G. Hüttmann, “Optical coherence tomography as an orientation guide in cochlear implant surgery?,” Acta oto-laryngologica, vol. 127, Sep. 2007, pp. 907-13.pdf
5. J.U. Kang and P. Gehlbach, “Endoscopic Functional Fourier Domain Common-Path Optical Coherence Tomography for Microsurgery,” IEEE Journal of Selected Topics in Quantum Electronics, vol. 16, Jul. 2010, pp. 781-792. pdf
6. B.J. Wong, J.F. de Boer, B.H. Park, Z. Chen, and J.S. Nelson, “Optical coherence tomography of the rat cochlea.,” Journal of biomedical optics, vol. 5, Oct. 2000, pp. 367-70. pdf
7. I. Fleming, M. Balicki, J. Koo, I. Iordachita, B. Mitchell, J. Handa, G. Hager, and R. Taylor, “Cooperative robot assistant for retinal microsurgery.,” Medical image computing and computer-assisted intervention : MICCAI … International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 11, Jan. 2008, pp. 543-50. pdf
8. Nakabayashi, Koki et. al. “OCT Optical Probe and Optical Tomography Imaging Apparatus”. US Patent Application No. 12/363,021. Filed 30 January 2009. pdf
9. C.G. Wright, P.S. Roland, and J. Kuzma, “Advanced bionics thin lateral and Helix II electrodes: a temporal bone study.,” The Laryngoscope, vol. 115, Nov. 2005, pp. 2041-5.pdf

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space.