Virtual Reality Drilling Simulator for Laminectomy: Implementing and Evaluating Colored Guidance

Last updated: 5/10/2023 8:12 AM

Summary

We are evaluating the use of a colored virtual-reality (VR) drilling navigation platform as part of pre-operative planning and surgical training for laminectomy and mastoidectomy surgeries. This image-guided navigation system is build on top of the AMBF platform and provides a dynamic color overlay for drilling that indicates safe anatomies (shown as green), sensitive structures that require caution (shown as yellow), and restricted structures (shown as red). We plan to assess clinical utility by conducting two user studies for laminectomy and mastoidectomy surgeries where subjects' surgical performance is compared against a control non-colored VR navigation platform. We also aim to incorporate new graphical user interface (GUI) features into the existing software, specifically a depth perception interface and a synchronized data extraction pipeline.

  • Students:
    • Kesavan Venkatesh: General Team Lead, BME Undergrad
    • Jonathan Wang: Clinical Lead, BME Undergrad
    • Yi Wang: Code Lead, Robotics MSE
  • Mentor(s):
    • David Usevitch: Postdoctoral Fellow LCSR
    • Hisashi Ishida: CS PhD Candidate
    • Adnan Munawar: Assistant Research Scientist LCSR

Figure I. VR Drilling System. (A) outlines the the various technical components & hardware comprising the VR drilling system. (B) presents the current setup of the system.

Note: project deliverables were changed over the course of the semester. These will be detailed in the following sections.

Background, Specific Aims, and Significance

Background

Laminectomy is a surgical procedure that involves removing portions of the lamina, a vertebral bone that forms the backside of the spinal canal covering the spinal cord. The procedure is generally recommended for patients suffering from spinal stenosis, and in the U.S. around 500,000 cases are performed annually [1]. Surgeons must be careful not to puncture past the dura (outer protective layer of spinal cord), but still incidental durotomy occurs in up to 11.3% [2].

Mastoidectomy is a surgical procedure performed to treat infections in the air-filled cells of the mastoid bone, and it is estimated that in the U.S. around 60,000 of these surgeries are performed annually [3]. However, the procedure carries significant risks, which includes the need for drilling into parts of the temporal bone to reach the internal auditory canal. This puts vital anatomical structures, such as the facial nerve and the semicircular canal, at risk of damage. Research has shown that the cognitive load for surgeons can peak at around 10% above average during the later, more complex stages of the surgery [4].

Prior work by PhD candidate Hisashi Ishida, Assistant Research Scientist Adnan Munawar, and Postdoctoral Fellow David Usevitch include a virtual drilling simulator using CSDN for training in laminectomy and mastoidectomy surgeries [5]. The simulator allows surgeons to control a virtual drill using a haptic device and provides a patient's anatomical volume created from CT scans for simulation. The system's guidance algorithm calculates the distance between the drill and the nearest anatomy and provides real-time feedback to the surgeon. Additionally, a novel algorithm based on SDFs of sensitive anatomical structures speeds up the computation of the closest anatomy.

Figure II. Visualization within VR Drilling System (A) presents the VR drilling system for mastoidecomy. (B) presents the VR drilling system for laminectomy.

  1. Red voxels indicate sensitive structures that should not be drilled
    1. Drilling red volumes results in a breach that flashes a red warning alert
  2. Yellow voxels indicate the user is drilling near a sensitive anatomy
    1. A yellow warning will recommend caution
  3. Green voxels are safe drilling regions, and no continuous warnings will be provided for drilling here
  4. Blue voxels are optional areas.

Specific Aims

The initial aims of this project were to validate the novel colored guidance VR drilling system with a clinical study.

Goal 1: Finish laminectomy user study
Goal 2: Perform a mastoidectomy feasibility study
Goal 3: Improve the GUI prior to implementation in mastoidectomy study.

However, midway through the project, our goals shifted towards completing the laminectomy user study and implementing new technical features for the simulator. This will be discussed later on.

Significance

Virtual-reality (VR) simulators for training in complex surgical procedures have demonstrated great great potential. Research has found that compared to non-VR training, VR training enhanced objective OR performance, decreased operation duration, and lowered the number of intraoperative errors [6]. VR systems have also found applications in pre-operative planning. Medical image data are manipulated to accurately plan surgery in a computer environment and then transfer that virtual plan to the patient using customized instruments.

We hope to contribute a validation study of a novel VR training platform for performing laminectomy and mastoidectomy surgeries to ultimately reduce surgical error rates and increase adoption of this technology.

Deliverables (ARCHIVED)

  • Minimum: (2/20 - 3/7)
    1. A reproducible written protocol for segmenting lumbar spine CT scans using 3D Slicer software [Complete]
    2. 15 locally-saved segmentation files of lumbar spines [Complete]
    3. Ready-to-use laptop with virtual-reality colored navigation GUI for laminectomy user study with the 15 lumbar spine cases [Complete]
  • Expected: (3/7 - 4/25)
    1. 10 x 16 locally saved hdf5 files of user data extracted for each virtual-reality session from Laminectomy User Study [Complete]
    2. Written results section for laminectomy paper for submission to IEEE Transactions on Medical Robotics and Bionics [Complete]
    3. Point-cloud visualization to replay user drilling [Complete]
    4. A program that visualizes phantom position in reachable workspace (input: phantom stylus coordinate, output: mini-GUI with marker indicating user depth in workspace) [Abandoned]
    5. 15 locally-saved segmentation files of mastoidectomy [Moved to Summer]
  • Maximum: (4/15 - 5/5)
    1. 10 x 16 locally saved hdf5 files of user data extracted for each virtual-reality session from Mastoidectomy Feasibility Study [Not Started]
    2. Written results section for mastoidectomy manuscript to be submitted to Otology and Neurotology [Not Started]
    3. Written first-authored draft for Otology and Neurotology evaluating the feasibility of a colored virtual-reality navigation system for mastoidectomy procedures [Not Started]
    4. A program that synchronizes data extraction with user recording during virtual-reality session (input: user pose and drilling progress, output: extracted data file) [Complete]
Activity Deliverables Status
Minimum Familiarize with 3D Slicer software and practice segmenting a lumbar spine CT scan A reproducible written protocol for segmenting lumbar spine CT scans using 3D Slicer software Complete
Segmenting lumbar spine CT scans following the written protocol 15 locally-saved segmentation files of lumbar spines Complete
Build virtual-reality platform locally and upload lumbar spine segmentations to GUI Ready-to-use laptop with virtual-reality colored navigation GUI for laminectomy user study with the 15 lumbar spine cases Complete
Expected Conduct 10 laminectomy user sessions at JHH and locally extract raw data from user recordings 10 x 16 locally saved hdf5 files of user data extracted for each virtual-reality session Complete
Performing data analysis and drafting results with mentor feedback Written results section for laminectomy paper for submission to IEEE Transactions on Medical Robotics and Bionics Complete
Discussions with ENT surgeons to determine relevant anatomy for mastoidectomy navigation and practice segmenting inner ear structures in 3D Slicer A reproducible written protocol for segmenting inner ear CT scans using 3D Slicer software Moved to Summer
Segmenting inner ear CT scans following the written protocol 15 locally-saved segmentation files of lumbar spines Moved to Summer
Build virtual-reality platform locally and upload inner ear segmentations to GUI Ready-to-use laptop with virtual-reality colored navigation GUI for mastoidectomy user study with the 15 inner ear cases Moved to Summer
Implement a depth perception interface A program that visualizes phantom position in reachable workspace (input: phantom stylus coordinate, output: mini-GUI with marker indicating user depth in workspace) Abandoned
Implement a playback video for user drilling A program that visualizes user drilling (input: removed voxel coordinates, output: mini-GUI with playback video) Complete
Maximum Conduct 4 mastoidectomy user sessions at JHH and locally extract raw data from user recordings 6 locally saved hdf5 files of user data extracted for each virtual-reality session Moved to Summer
Performing data analysis and drafting results Written results section for mastoidectomy paper for submission to Otology and Neurotology Not Started
Literature review and discussions with mentors and ENT surgeons Written first-authored draft for Otology and Neurotology evaluating the feasibility of a colored virtual-reality navigation system for mastoidectomy procedures Not Started
Implement a C++ plugin for data extraction A program that synchronizes data extraction with user recording during virtual-reality session (input: user pose and drilling progress, output: extracted data file) Complete

Milestones and Status (ARCHIVED)

Phase Milestone Exit Criteria Start Date End Date Status
Phase 1: Laminectomy User Study Spine CT segmentation Five lumbar spines saved as 15 segmentation files of CT scans of L1-L3 regions 2/20 3/5 Completed
Colored VR platform setup VR platform with GUI that loads segmented spines with color overlay 3/6 3/12 Completed
Finalize user study protocol applicable for both laminectomy and mastoidectomy user studies Written and mentor-approved protocol outlining data collection and analysis 2/20 2/26 Completed
Schedule laminectomy user studies Confirmed sessions with >= 10 subjects for laminectomy user study and planned monitoring assignments for team members 2/27 3/19 Ongoing
Collect surgeon data and perform data analysis Generate figures for comparison of drilling methods, NASA-TLX survey results, and surgical learning 3/20 4/16 Ongoing
Phase 2: Mastoidectomy Feasibility Study Inner Ear CT segmentation 15 segmentation files of CT scans of inner ear anatomy 3/13 3/19 Ongoing
Colored VR platform setup VR platform with GUI that loads segmented ears with color overlay 3/20 3/26 Ongoing
Schedule mastoidectomy user studies Confirmed sessions with >= 10 subjects for mastoidectomy user study and planned monitoring assignments for team members 3/27 4/9 Not Started
Collect surgeon data and perform data analysis Generate figures for comparison of drilling methods, NASA-TLX survey results, and surgical learning 4/10 4/30 Not started
Draft mastoidectomy feasibility paper for Otology and Neurotology Generate results figures for comparison of drilling methods, NASA-TLX survey results, and surgical learning 5/1 5/15 Not started
Phase 3: VR GUI Development Depth perception interface Functional, unit-tested, and documented Python script that builds a workspace model, captures the current phantom pose as input, and marks the relative position 2/27 3/19 Abandoned
Synchronized C++ data extraction plugin A C++ package that captures user pose and drilling progress from ROS and outputs extracted data metrics 4/3 4/30 Not started

PIVOT IN DELIVERABLES & PHASES (4/10)

Over the course of this project, our deliverables were adjusted based on delays in critical dependencies. Most notably scheduling participants and coordinating logistics with the Orthopedic department took much longer than anticipated. After our checkpoint presentation, our proposed deliverables and their status are as follows. We have finished all of our minimum and expected deliverables.

Activity Deliverables Status
Minimum Familiarize with 3D Slicer software and practice segmenting a lumbar spine CT scan A reproducible written protocol for segmenting lumbar spine CT scans using 3D Slicer software Complete
Segmenting lumbar spine CT scans following the written protocol 15 locally-saved segmentation files of lumbar spines Complete
Build virtual-reality platform locally and upload lumbar spine segmentations to GUI Ready-to-use laptop with virtual-reality colored navigation GUI for laminectomy user study with the 15 lumbar spine cases Complete
Expected Conduct 8 laminectomy user sessions at JHH and locally extract raw data from user recordings 8 locally saved hdf5 files of user data extracted for each virtual-reality session Complete
Recruit and conduct 3 more laminectomy user sessions at JHH and locally extract raw data from user recordings 3 locally saved hdf5 files of user data extracted for each virtual-reality session Complete
Performing data analysis Figures for manuscript comparing user performance under different guidance methods Complete
Drafting results section for manuscript Written results section for laminectomy paper for submission to IEEE Transactions on Medical Robotics and Bionics Complete
Implement a playback video for user drilling A program that visualizes user drilling (input: removed voxel coordinates, output: mini-GUI with playback video) Complete
Maximum Implement a C++ plugin for data extraction A program that synchronizes data extraction with user recording during virtual-reality session (input: user pose and drilling progress, output: extracted data file) Complete

Haptic Device Visualization (ABANDONED)

Note this portion of technical development was abandoned in favor of pursuing other technical features were recommended by our mentors Hisashi and Adnan. Further technical developments are highlighted later.

Currently, users benefit from haptic feedback from the stylus to determine their relative position and what anatomy they are touching in the virtual-reality world. However, the physical workspace has a constrained dimension, and users often get confused when the stylus hits the phantom base itself, thinking that the perceived force feedback is part of the virtual-reality world. We’d like to incorporate a design interface meant to warn users when the stylus is about to hit the phantom base.

We plan to do this by using the coordinates of the stylus relative to the phantom base [7]. Consider the phantom base to be the origin. We ask users to calibrate the system by extending their stylus as far out in front of the base as they can. This establishes the z-axis depth. We will empirically set a z distance threshold t to the x-y plane, wherein if the stylus is within that threshold the marker will turn red indicating impending contact with the phantom base.

Figure III: Depth Perception Schematic of depth perception interface with top row being the physical workspace and bottom row depicting the intended GUI interface. (A) Stylus is not within contact threshold with base. (B) Stylus is within contact threshold with base. Marker turns red when near the threshold.

Clinical User Study

Background

The study proposed to examine and improve methods of augmented drill navigation in surgery, specifically performing laminectomies. Drilling in these spine surgeries requires precision drilling in a specific area and to a specific depth. Delicate anatomy such as the spinal cord needs to be avoided, and often a 1 mm layer is kept in the superior half, protecting the vertebral foramen anatomy, which the spine surgeon can easily break off when removing bone. Additionally, enough bone needs to be preserved so as to preserve the structural integrity of the spine segment. Drilling for these surgeries can be particularly challenging and taxing on surgeons, and various navigation methods have been developed to improve drilling. These include audible noises and speech, colors to direct drilling regions, and similar. This study will test several different navigational cues, or combinations of cues, to participants as the subject to assist in the drilling task.

Please feel free to reference our System Test Plan Document for more information.

IRB Approval

The user study proposed in this document was submitted to the Johns Hopkins Internal Review Board (IRB) under the title “Virtual Laminectomy Drilling Comparing Colored Voxel Region Representations with Audio/Visual Warnings using a Head-Mounted Display” under principal investigator Dr. Amit Jain. The application was approved on March 16, 2023 with the IRB number IRB00351495. The IRB application was revised on April 18, 2023 to allow for the inclusion of the NASA Task Load Index (NASA-TLX) Survey.

As a part of the IRB study, all team members completed the Researchers (CITI), Human Subjects Research – Biomedical Research (CITI), Conflict of Interest and Commitment. No personal identifiers were collected at any point throughout the study.

Proposed Design

Figure IV: Clinical Study Design Outline Here we outline our clinical study design for evaluating the virtual guidance system for laminectomy & mastoidectomy. The first phase is named the Practice Phase as participants are given two segmentations to familiarize themself with non-colored v.s. colored segmentations and the overall virtual guidance system. No outcomes of interest are measured during this phase. Outcomes are measured during all the subsequent phases. The next phase is the Evaluation Phase, where participants are to work through 13 segmentations, both non-colored and colored. Colored and non-colored guidance effectiveness is evaluated with measured outcomes during this phase.

We have scheduled user studies at the hospital in collaboration with Dr. Micheal Raad and Dr. Rachel Bronheim. They are scheduled for 3/30 and 4/06. We are going to setup our simulator within the Johns Hopkins Outpatient Center.

Post-Session Questionnaires

Following the measured cases, participants were asked to complete a short questionnaire about the drilling and navigation methods which were tested. This questionnaire will ask about the effectiveness of the drilling platforms tests and participants' preferences to these systems. The questionnaire will also ask if participants are a medical professional or not. If so, it will ask how many years the participants have been a practicing medical professional, and if participants perform drilling procedures on a frequent basis. This questionnaire allows for the collection of feedback about the system and additionally provides opportunities to stratify participants based on surgical experience.

The NASA-TLX survey was also provided to participants after mending the IRB protocol. The survey allowed for the quantification of cognitive load experienced by participants by having participants rate the following measures on a scale of 1 to 20: Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, and Frustration.

Note: these forms can be found in documentation at the bottom of this wiki

Re-Segmentations and Shading Fixes

We generated new segmentations of the lumbar spine with the guidance of Dr. Jain. These new segmentations are more clinically accurate than the ones we have been operating with beforehand. This new segmentation presented challenges with EDT / shader generation though, so we took some time debugging the code to successfully generate the desired shading. Figure V: Re-Segmented Lumbar Spines We fixed the old segmentations to new ones based off of Dr. Jain's recommendations.

Figure VI: Modified EDT Generation Fixed the buggy EDT generation.

Synchronized Data Recording

We proposed and outlined a synchronous method for AMBF data extraction. Instead of using the GUI to set the recording mode, we would modify the keyboard_update() function inside ./plugin/volumetric_drilling/volumetric_drilling.cpp to toggle the recording mode based on a start and stop hotkey. Then, inside the physics_update() method, the H5DataWriter class will store the color, coordinates, and time of each voxel that is removed. This class will write to a local hdf5 file with a tunable buffer. By natively writing to HDF5 files inside the C++ plugin, we would eliminate the need for publishing and querying simulation data asynchronously over a ROS server. Obviating this multithreading would lower the computational cost of the simulator on the PC.

Figure VII: Proposed HDF5 Saving C++ Plugin

Figure VIII: Flowchart for HDF5 Saving C++ Plugin

Visualized Playback of User Drilling

Figure IX: Visualized Time Lapse of a User Can help show colored voxels drilled (with positions and timings).

After conducting a VR session, surgeons need a way to visualize their drilling to receive performance feedback. The current simulator saves HDF5 files but does not include an expressive way to visualize them. We developed a replay system that creates a time-lapse video of the drilling (complete with colors and drill kinematics).

Implementing Signed-Distance Fields

SDFs are the backbone for the color-guidance implemented in the simulator. Different bone pieces are colored based on their relative distance to sensitive anatomies; these distances are queried from the SDF for each anatomy of interest. The SDF itself is computed from the 3D volume extracted from 3D Slicer.

Theory: We detail the mathematical details of computing an SDF in a 2D grid. (These results naturally extend to 3D volume.) An SDF grid can be calculated using a 2-step transform [14]. Figure 7 models this approach. The blue squares represent the sensitive anatomical structure and the yellow squares are the surrounding regions in the grid (in 3D, voxels in the CT scan). The first transform computes the minimum horizontal distance to the surface of the anatomy, and the second transform factors in vertical direction to compute the minimum overall squared distance to the anatomy. Equation 1 specifies these transforms.

Figure X: EDT Example Toy example of EDT applied to a 2D grid. Blue pixels are anatomy, yellow pixels are surrounding. Final grid has each square assigned a number indicating minimum squared distance to the surface of the anatomy. Note also that the anatomy can be a surface, i.e., have yellow pixels inside of it, but this is not shown for simplicity.

Equation 1: Two-step transform in the 2D Euclidean Transform used to compute the SDF grid.

Implementation in VR Simulator: Surgeons need a color-guidance that is not only layered depth-wise but also provides color boundaries in the plane of the screen. Therefore we decided to build our SDF volume based on three individual SDFs: the vertebral foramen (VF) behind the lamina (since this is the spinal cord and is strictly off limits for drilling), bone surrounding the lamina to be drilled, and bone in the lumbar vertebra beneath the lamina. A voxel’s signed distance in the composite was the minimum across the three SDFs (Equation 2).

Figure XI: Expanded View of Colored Boundaries and Distance Thresholds

Equation 2: Composite SDF computed based on signed distances from the VF

Each anatomy's SDF volume was created and stored offline as a stack of 2D images; these were generated using the ./scripts/EDTImageGeneration/EDTImageGenerationCmdLine workflow. In ./plugin/volumetric_drilling/volumetric_drilling.cpp, we created three variables to load each SDF volume specified. However, in OpenGL, the shader was implemented to handle only two SDFs (one for the VF, one for the bone). This required us to combine the two bone SDFs offline using the ./scripts/EDTImageGeneration/combine_edt_images.py script.

Results & Analysis

The completion percentage and number of unintended voxels removed are reported below. While there was no significant difference in the number of unintended breaches made between the baseline and color-guided conditions, providing a color overlay significantly increased the mean drilling accuracy from 31% completion to 47% (p<0.05).

Figure XII: Average Completion Percentage and Number of Unintended Voxels Removed

The drilling times under both conditions were compared. There was no significant difference found between baseline and color-guided conditions. Figure XIII: Drilling time did not significantly change between color-guidance and baseline conditions

Survey results from the NASA-TLX assessment were collected for 7/11 participants (Figure 13). Color-guidance significantly reduced the mental demand (14.43 vs. 9.14, p<0.05), required effort (16.86 vs. 11.86, p<0.05), and frustration with the VR platform (12.86 vs. 8.14, p<0.01).

Figure XIV: NASA-TLX survey results indicated reduced cognitive load with color-guidance than without.

Discussion

Our user study focused on evaluating the impact of color-guidance on drilling accuracy and operation time. Through the use of AMBF for quantitative data collection and gathering qualitative feedback from participant drilling sessions, we obtained valuable insights.

The comparison of drilling accuracy revealed that participants successfully avoided sensitive anatomies equally well in both conditions. However, with color guidance, they tended to remove more of the required lamina. This can be attributed to participants having a better intuition about where not to drill, such as avoiding the spinal cord. The color-guidance system proved particularly helpful in indicating how much drilling was safe. Our collaborators at JHMI Orthopedic Surgery provided anecdotal evidence suggesting that there are varying standards regarding the appropriate depth-wise removal of the lamina. Hence, the color-guided VR platform has the potential to serve as a standardized educational tool for resident surgeons specializing in laminectomies.

Interestingly, the total drilling times did not show a significant difference between the baseline and color-guided conditions. These findings align with previous results obtained using the AMBF-based simulator and suggest that the platform may be better suited for improving surgical accuracy rather than speed.

Future Work

We intend to submit our findings and the technical approach of color-guidance to IEEE Transactions on Medical Robotics and Bionics.

The valuable insights obtained from this study have shaped our future project directions. As a result, our immediate focus is on integrating the synchronous data extraction code into the platform. This implementation aims to reduce the CPU cost associated with data recording, thereby optimizing the overall efficiency of the system.

Management Summary

We are a team of three engineers with multidisciplinary backgrounds in software development, engineering design, robotics, and medicine. We leveraged these complementary skill-sets effectively in our team through the following management structure.

Logistics

We met 1-2 times a week (at either Brody Learning Commons or Zoom, depending on availability/progress). We regularly met with mentors (Hisashi: weekda evenings depending on schedules, David: Wednesday mornings) relay progress and ask questions.

We conducted our user study sessions at the Johns Hopkins Outpatient Center on Thursday mornings throughout the month of April. In total, we completed 4 sessions with 11 participants.

We used Microsoft Teams as the primary communication platform. Software development was done collaboratively on GitHub. Write-ups/reports were be drafted in Google Docs and submitted formally using LaTeX. Presentations were made in Google Sheets.

Distribution of Work

Kesavan Venkatesh

  1. Developed and built VR system with color-guidance
  2. Revised spine segmentations and uploaded to simulator
  3. Coordinated with resident orthopedic surgeons to schedule user studies
  4. Outlined C++ plugin for synchronized data extraction

Jonathan Wang

  1. Developed and built VR system with color-guidance
  2. Revised spine segmentations and uploaded to simulator
  3. Proctored user studies and helped organize user study logistics
  4. Outlined C++ plugin for synchronized data extraction

Yi Wang

  1. Extract the data of user study and convert the file from hdf5 to csv
  2. Analyzed user study data and reported statistics for voxels removed and drilling time
  3. Developed a GUI to replay drilling sessions

Overall Progress

In our initial proposal, we planned to recruit 16 participants in our user study. Due to delays in acquiring IRB approval and difficulties setting up communication with orthopedic surgeons, we were only able to recruit 11. Furthermore, IRB delays limited us to only collect NASA-TLX survey results from 7/11 participants.

These delays in our user study compounded into delays with our technical development. We ran out of time to fully unit-test and integrated our synchronized C++ plugin. This is an immediate next step we are planning for in the summer, since we saw in our user studies how valuable a native C++ data extraction scheme would be in reducing the CPU load of the simulator.

Lessons Learned

As there was a heavy clinical focus for the project, group members dived deep into learning the ins and outs of laminectomy. From exploring the various anatomies of the spine (L1, L2, L3, spinous processes, etc.) to consulting with surgeons on what realistic laminectomies look like, we truly gained a newfound appreciation for understanding the clinical relevance and motivation for a engineering project. 3D Slicer was also a valuable software to gain expertise on, and members are now capable to to generate custom and modified segmentations given CT scans of anatomies.

Planning and conducting the user study was also a huge legwork. Initially we had thought as long as we dedicated our time to going to the hospital, there would be participants lined up ready and we could finish collecting data in just a four weeks. But after our user sessions, we have come to understand how intricate scheduling and timing can be, especially when working with medical professionals who already have more than enough on their plates. Residents would often have to take a 10-20 minute break in the middle to go check up on a case, which would have a cascading effect on future participants' schedules. There were specific user sessions where we only came out with having collected only one participant's data. User studies are much hard to conduct and see though, but whether it be reaching out to residents on their “research years” to writing an entire protocol with script for conducting each session, we gained invaluable experience into the inner workings of conducting a proper user study. Looking forward into the future, we can truly look back upon this experience to help guide us in carrying out well thought-out user studies to evaluate innovative technologies.

Dependencies

Dependency Need Status Contingency Plan Planned Hard Resolved
Full IRB approval for user study at JHH Organize and execute user study Resolved N/A 3/6 3/12 3/17
Github access Locally build VR system Resolved N/A 2/20 2/20 2/20
CT scans & prior segmentations for laminectomy Segment anatomy according to ENT surgeon recommendations Resolved N/A 2/20 2/20 2/20
3D Slicer Make segmentations Resolved N/A 2/20 2/20 2/20
Linux machine and VR glass Locally running VR simulator for laminectomy and mastoidectomy studies Resolved Use lab’s VR glasses and (if needed) borrow a linux laptop 3/6 3/12 3/13
Access to hospital Collecting clinical study data Resolved N/A 3/6 3/12 3/8
Access to drafted manuscript and IRB documents for laminectomy study Review prior user study protocols Resolved N/A 2/20 2/20 2/20
Swipe access to LCSR PhD Office Access to VR glasses Resolved Yi enter PhD Office 2/25 3/3 3/1
Study participants Participate in clinical study Resolved N/A 3/5 3/17 4/6

Milestones and Status

Phase Milestone Exit Criteria Start Date End Date Status
Phase 1: Laminectomy User Study Spine CT segmentation Five lumbar spines saved as 15 segmentation files of CT scans of L1-L3 regions 2/20 3/5 Completed
Colored VR platform setup VR platform with GUI that loads segmented spines with color overlay 3/6 3/12 Completed
Finalize user study protocol applicable for both laminectomy and mastoidectomy user studies Written and mentor-approved protocol outlining data collection and analysis 2/20 2/26 Completed
Schedule laminectomy user studies Confirmed sessions with >= 10 subjects for laminectomy user study and planned monitoring assignments for team members 2/27 3/19 Completed
Collect surgeon data and perform data analysis Generate figures for comparison of drilling methods, NASA-TLX survey results, and surgical learning 3/20 4/16 Completed
Phase 2: VR GUI Development Depth perception interface Functional, unit-tested, and documented Python script that builds a workspace model, captures the current phantom pose as input, and marks the relative position 2/27 3/19 Abandoned
Synchronized C++ data extraction plugin A C++ package that captures user pose and drilling progress from ROS and outputs extracted data metrics 4/3 5/15 In Progress
Visualization of playback for removed voxels A python script that generates a .mp4 video that plots the point cloud of removed voxels 4/12 4/23 Completed

Reports and presentations

Project Bibliography

[1] K. Phan, R. J. Mobbs, Minimally Invasive Versus Open Laminectomy for Lumbar Stenosis: A Systematic Review and Meta-Analysis, Spine 41 (2) (2016) E91–E100. doi:10.1097/BRS.0000000000001161.

[2] Z. Li, G. Yu, S. Jiang, L. Hu, W. Li, Robot-assisted laminectomy in spinal surgery: a systematic review, Annals of Translational Medicine 9 (8) (2021) 715. doi:10.21037/atm-20-5270.

[3] B. T. Jankowitz, D. S. Atteberry, P. C. Gerszten, P. Karausky, B. C. Cheng, R. Faught, W. C. Welch, Effect of fibrin glue on the prevention of persistent cerebral spinal fluid leakage after incidental durotomy during lumbar spinal surgery, European Spine Journal: Official Publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society 18 (8) (2009) 1169–1174. doi:10.1007/s00586-009-0928-6

[4] M. Bydon, M. Macki, N. B. Abt, D. M. Sciubba, J.-P. Wolinsky, T. F. Witham, Z. L. Gokaslan, A. Bydon, Clinical and surgical outcomes after lumbar laminectomy: An analysis of 500 patients, Surgical Neurology International 6 (Suppl 4) (2015) S190–193. doi:10.4103/2152-7806.156578.

[5] H. Ishikura, S. Ogihara, H. Oka, T. Maruyama, H. Inanami, K. Miyoshi, K. Matsudaira, H. Chikuda, S. Azuma, N. Kawamura, K. Yamakawa, N. Hara, Y. Oshima, J. Morii, K. Saita, S. Tanaka, T. Yamazaki, Risk factors for incidental durotomy during posterior open spine surgery for degenerative diseases in adults: A multicenter observational study, PloS One 12 (11) (2017) e0188038. doi:10.1371/journal. pone.0188038.

[6] L. C. French, M. S. Dietrich, R. F. Labadie, An estimate of the number of mastoidectomy procedures performed annually in the United States, Ear, Nose, & Throat Journal 87 (5) (2008) 267–270

[7] F. Heinrich et al., “HoloPointer: a virtual augmented reality pointer for laparoscopic surgery training,” Int J Comput Assist Radiol Surg, vol. 16, no. 1, pp. 161–168, Jan. 2021, doi: 10.1007/s11548-020-02272-2.

[8] M. Estefan, S. Munakomi, and G. O. Camino Willhuber, “Laminectomy,” in StatPearls. Treasure Island (FL): StatPearls Publishing, 2023. [Online]. Available: http://www.ncbi.nlm.nih.gov/books/NBK542274/

[9] J. Y. Du, A. Aichmair, J. Kueper, C. Lam, J. T. Nguyen, F. P. Cammisa, and D. R. Lebl, “Incidental Durotomy During Spinal Surgery: A Multivariate Analysis for Risk Factors,” Spine, vol. 39, no. 22, p. E1339, Oct. 2014.

[10] W. C. McGaghie, S. B. Issenberg, E. R. Cohen, J. H. Barsuk, and D. B. Wayne, “Does Simulation-Based Medical Education With Deliberate Practice Yield Better Results Than Traditional Clinical Education? A Meta-Analytic Comparative Review of the Evidence,” Academic Medicine, vol. 86, no. 6, p. 706, Jun. 2011.

[11] M. A. Aziz, J. C. Mckenzie, J. S. Wilson, R. J. Cowie, S. A. Ayeni, and B. K. Dunn, “The human cadaver in the age of biomedical informatics,” The Anatomical Record, vol. 269, no. 1, pp. 20–32, 2002, eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/ar.10046. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/ar. 10046

[12] K. W. van Dongen, E. Tournoij, D. C. van der Zee, M. P. Schijven, and I. A. M. J. Broeders, “Construct validity of the LapSim: Can the LapSim virtual reality simulator distinguish between novices and experts?” Surgical Endoscopy, vol. 21, no. 8, pp. 1413–1417, Aug. 2007. [Online]. Available: https://doi.org/10.1007/s00464-006-9188-2

[13] G. Reddy-Kolanu and D. Alderson, “Evaluating the effectiveness of the Voxel-Man TempoSurg virtual reality simulator in facilitating learning mastoid surgery,” Annals of The Royal College of Surgeons of England, vol. 93, no. 3, pp. 205–208, Apr. 2011. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3291135/

[14] R. Leuwer, A. Petersik, B. Pflesser, A. Pommert, B. Tolsdorff, K. H. H ̈ohne, and U. Tiede, “VOXEL-MAN TempoSurg A Virtual Reality Temporal Bone Surgery Simulator,” JOURNAL OF JAPAN SOCIETY FOR HEAD AND NECK SURGERY, vol. 17, pp. 203–207, Jan. 2007.

[15] M. Pfandler, M. Lazarovici, P. Stefan, P. Wucherer, and M. Weigl, “Virtual reality-based simulators for spine surgery: a systematic review,” The Spine Journal, vol. 17, no. 9, pp. 1352–1363, Sep. 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S1529943017302085

[16] T. Saito and J.-I. Toriwaki, “New algorithms for euclidean distance transformation of an n-dimensional digitized picture with applications,” Pattern Recognition, vol. 27, no. 11, pp. 1551–1565, Nov. 1994. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ 0031320394901333

Other Resources and Project Files

Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space (2023-19).

Colored SDF Github Repository Data Analysis Github Repository

courses/456/2023/projects/456-2023-19/project-19.txt · Last modified: by kvenka10




ERC CISST    LCSR    WSE    JHU