======Improve Content Validity of Virtual Drilling Simulator====== **Last updated: 11th May 2023 on 7:48 am** ======Summary====== Surgical simulators can assist surgeons on developing the surgical skills and spatial perception needed to perform an operation without the risk of accidents occurring on actual patients. This project aims to find a relation between force applied during drilling, sound of the drill and density of the material being drilled into. * **Students:** Anushruti Singh * **Mentor(s):** Hisashi Ishida, Adnan Munawar, Prof. Peter Kazanzides, Dr. Deepa Galaiya {{ :courses:456:2023:projects:456-2023-14:virtual_drilling_simulator_2_.png?500 |}} fig 1: Virtual Drilling Simulator {{ :courses:456:2023:projects:456-2023-14:fivrs.png?400 |}} fig 2: Fully Immersive Virtual Reality Simulators or FIVRs ======Background, Specific Aims, and Significance====== Auditory feedback has great potential in surgical simulators that aim at training surgeons associated to the correct interpretation of anatomy from sounds. There have been various studies on how anatomy especially bones can be differentiated by using the sounds during drilling. FIVRS or Fully Immersive Virtual Reality System was introduced previously which combined simulation software with hardware setup to provide realistic training for skull-based surgeries. Specific aims of this project are - - Collect a data set. This data set should contain all features needed to model the force produced by the drill, the audio during the drilling process and density of the phantom that was drilled on. - Model collected data to find relations between the varying parameters. - Implement an algorithm into the virtual-drilling simulator (FIVRS) {{ :courses:456:2023:projects:456-2023-14:screenshot_259_.png?700 |}} ======Deliverables====== * **Minimum:** (Expected by: 15 April 2023) 1. Create and utilize existing phantoms and collect sound, density and force data while drilling. * **Expected:** (Expected by: 22 April 2023) 2. Model the data of sound * **Maximum:** (Expected by: Not during this semster) 3. Implementation report of the functionality in the drilling simulator based on the modeled data. ======Technical Approach====== The technical approach is divided into three major categories - ===== Data Collection ===== : Data set will be collected on 3 phantoms of different measured densities of dental stone. Drilling will be done with certain constraints to the environment such as : a. Constant RPM of the drill b. Constant angle of drilling c. Constant speed of drilling The collected data will be modeled to relate the sound of the drill to the force. Phantom Prep: Three phantoms were used in the process of data collection. All the phantoms were similar in shape and size. The densities of these phantoms were varied. To mimic bone, the first and second phantoms had the same density as the extreme ends of the density scale of bone(cancelous and cortical). These values are 1178 g/m3 and 2100 g/m3. Average of these two values was used as the density for the third phantom. These phantoms were made using dental stone. ^ Density(kg/m3) ^ Minimum ^ Maximum^ |Bone (Cancelous)|1080|1350| |Bone (Cortical)|1800|2100| Table: 1 - Density. Density " IT'IS Foundation. (n.d.). Retrieved March 29, 2023, from https://itis.swiss/virtual-population/tissue-properties/database/density ^Phantom ^Density(kg/m3)^ |Phantom 1|1084.065| |Phantom 2|2100.3| |Phantom 3|1630.6| Table 2: Phantom densities {{ :courses:456:2023:projects:456-2023-14:phantom.jpeg?300 |}} fig 3: Initial Phantom {{ :courses:456:2023:projects:456-2023-14:screenshot_223_.png?300 |}} fig 4:Final Phantom (mounted onto gamma sensor) ====== Experimental Setup: ====== ===== Lab Setup: ===== The audio file was recorded with a Logitech Yeti X microphone. The microphone was placed on a table along with the gamma sensor. The gamma sensor was mounted with the phantom and fixed with the help of four bolts. An Anspach force-sensing drill was fixed onto the Galen robot. The arm was moved such that the drill bit was positioned over the phantom. {{ :courses:456:2023:projects:456-2023-14:picture6.jpg?400 |}} fig 5: Experimental setup for data collection ====== Modelling Data ====== The data set collected will be used to model relations between the forces produced during drilling, the sounds produced during drilling, and the densities of the phantom being drilled on. ===== Data Synchronization ===== During the recording, the phantom and the microphone were tapped manually before the drilling started. These taps were reflected as two spikes in both the audio and force data. The peak of the first spike was used to synchronize the data. {{:courses:456:2023:projects:456-2023-14:picture4.png?200|}} {{:courses:456:2023:projects:456-2023-14:picture5.png?200 |}} fig 6: Taps as visualised in the audio and force data. ===== Signal Processing: ===== After synchronization, samples of 8.04 seconds of drilling sound were extracted from both the wav files and the rosbag files. There was a total of thirty samples, ten from each phantom. To remove the background noise, these samples were passed through a high pass filter with the cut-off frequency set at 8,000Hz. To study the effect of density on the audio, single-sided amplitude spectrum plots were studied. The offset in the wrench-force samples was fixed by shifting the entire sample by the minimum value of that sample. The resultant of the wrench-force samples was calculated and plotted. These results will be further discussed in the results section. {{ :courses:456:2023:projects:456-2023-14:picture1.jpg?400 |}} fig 7: High pass filter applied on audio wav files ====== Results: ====== There was a total of eight samples for LL, seven samples for Avg, and seven samples for UL. ===== Audio Analysis: ===== It was observed that the amplitude of the frequencies increased as the density of the phantom was increased. {{ :courses:456:2023:projects:456-2023-14:picture2.png?600 |}} fig 8: Single-sided amplitude graphs for LL3, AVG10, and UL7 The highest amplitudes for the LL, Avg, and UL phantoms were observed at 0.0007, 0.00099, and 0.00103 units respectively. There was a prominent increase in amplitude for the 8000 to 16000 Hz region. ===== Force Analysis: ===== For, the gamma sensor, was observed that all wrench-force plots had a sudden increase in the start followed by a steady decline which eventually reduced to zero. The sudden increase is associated with the drill drilling down into the phantom. It was observed that the wrench-force value increased with the increase in the density of the phantom. {{ :courses:456:2023:projects:456-2023-14:picture3.png?600 |}} fig 9: Force analysis for wrench-force data (Wrench-force measured in N) With an increase in density, the max wrench-force value increases from 0.0146 N in LL to 0.033692 N for Avg and 0.1610 N in UL. ======Dependencies====== ^ Dependency ^ Need ^ Status ^ Follow-up ^ Contingency Plan ^ Deadline ^ |Force-sensing Drill|Force data collection|Acquired|N/A|Obtain another force-sensing drill that can fit on the Galen Robot|21st Feb| |Basic microphone|Sound data collection|Acquired|N/A|Use mobile's mic to record sound|21st Feb| |Phantoms|To drill onto for data collection|Acquired|N/A|Request more phantoms to be printed|21st Feb| |Access to MockOR| To work on the project|Acquired|N/A|Work on the project under supervision|23rdFeb| |Permission to use Galen Robot|To control the operation of the drill|Acquired|N/A|N/A|28th Feb| |Computer with ROS, Linux and Python|Modelling and implementation of function|Acquired|N/A|Use Virtual Machine on laptop|28th Feb| Without the Galen robot or the force-sensing drill, the project would come to a complete halt. All permissions have been collected. ======Milestones and Status ====== - Milestone name: Data Set Collected * Planned Date: 3 April 2023 * Expected Date: 3 April 2023 * Status: Completed 3 April 2023 - Milestone name: Model the collected data * Planned Date: 15 April 2023 * Expected Date: 20 April 2023 * Status: In-progress - Milestone name: Implementation of the function into the simulator * Planned Date: 20 May 2023 * Expected Date: xxx * Status: Not started ======Reports and presentations====== * Project Plan * {{ :courses:456:2023:projects:456-2023-14:project_plan_presentation.pdf |}} * {{ :courses:456:2023:projects:456-2023-14:project_proposal.pdf |}} * Project Background Reading * See Bibliography below for links. * {{ :courses:456:2023:projects:456-2023-14:background_reading_summary_report.pdf |}} * {{ :courses:456:2023:projects:456-2023-14:background_reading_presentation.pptx |}} * Project Checkpoint * {{ :courses:456:2023:projects:456-2023-14:checkpoint_presentation_final.pptx |}} * Project Final Presentation *{{ :courses:456:2023:projects:456-2023-14:cisii_poster.pdf |}} * Project Final Report * {{ :courses:456:2023:projects:456-2023-14:final_report.pdf |}} ======Project Bibliography======= * __Paper for background reading report__: Zakeri, V., & Hodgson, A. J. (2017). Classifying hard and soft bone tissues using drilling sounds. 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). doi:10.1109/embc.2017.8037452 * Bone Density. Density " IT'IS Foundation. (n.d.). Retrieved March 26, 2023, from https://itis.swiss/virtual-population/tissue-properties/database/density/ ===== References ===== 1. Hoffmann, P.F., Gosselin, F., & Taha, F. (2009). Analysis of the drilling sound component from expert performance in a maxillo-facial surgery. 2. Munawar, A., Li, Z., Kunjam, P., Nagururu, N., Ding, A. S., Kazanzides, P., Looi, T., Creighton, F. X., Taylor, R. H., & Unberath, M. (2021). Virtual reality for synergistic surgical training and data generation. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 10(4), 366–374. https://doi.org/10.1080/21681163.2021.1999331 3. BOESNACH, I., HAHN, M., MOLDENHAUER, J., BETH, T. H., & SPETZGER, U. (2004). Analysis of drill sound in spine surgery. Perspective in Image-Guided Surgery. https://doi.org/10.1142/9789812702678_0011 4. BOESNACH, I., HAHN, M., MOLDENHAUER, J., BETH, T. H., & SPETZGER, U. (2004). Analysis of drill sound in spine surgery. Perspective in Image-Guided Surgery. https://doi.org/10.1142/9789812702678_0011 5. Chen, X., Sun, P., & Liao, D. (2018). A patient-specific haptic drilling simulator based on virtual reality for dental implant surgery. International Journal of Computer Assisted Radiology and Surgery, 13(11), 1861–1870. https://doi.org/10.1007/s11548-018-1845-0 6. Munawar, A., Li, Z., Nagururu, N., Trakimas, D., Kazanzides, P., Taylor, R.H., & Creighton, F.X. (Sent for approval to IPCAI). Fully Immersive Virtual Reality for Skull-base Surgery: Surgical Training and Beyond. 7. Bone Density. Density " IT'IS Foundation. (n.d.). Retrieved March 26, 2023, from https://itis.swiss/virtual-population/tissue-properties/database/density/ ======Other Resources and Project Files====== Phantom Model CAD file: https://livejohnshopkins-my.sharepoint.com/:f:/g/personal/asing119_jh_edu/EjcHsydOePZAgZP4_rkOffMBhElDdtH23CpHNLIKw1_Sbw?e=PJ3fFM Data Collection Document: {{ :courses:456:2023:projects:456-2023-14:data_collection_record.pdf |}} Code files: https://drive.google.com/drive/folders/1vDTytD8-F91ExqGHsYcKsO-DfSpX7div?usp=sharing