Last updated: 4/29/2022

The ultrasound probe is a piece of vital equipment for ultrasound diagnosis. However, the current ultrasound test needs operators to hold the probe, which is inconvenient for an expert to conduct other actions in the meantime. This project aims to build a hand-free patch ultrasound that could be controlled by the expert using the user interface (UI).

• Students: Shuran Zhang, Yuanwu He
• Mentor(s): Keshuai Xu, Emad Boctor

Background and Aims

Ultrasonography has been widely used in clinical practice for decades, but the devices used for diagnoses are usually huge. Even though portable ultrasound devices have been invented in recent years, they all need the operator to hold the probe to detect the patient’s body. Hence, the project wants to build a new generation of medical ultrasound called patch ultrasound, which has the benefits of:

1. Hands-free probe.
2. Users can acquire the ultrasound image from multiple angles.
3. It could be controlled by the expert remotely.

This project aims to design a patch that has the above three advantages. Ideally, the created prototype should be able to hold the probe so that the expert is not required to hold it when making the diagnosis. Furthermore, since the expert is not holding the probe, the prototype should have the function of changing the detecting angle so that the expert can view the ultrasound images thoroughly. Finally, the project also needs to design the UI so that the prototype would be eligible for remote control, through which the expert could diagnose the patient without being in the flesh.

Significance

Patch ultrasound is eligible for fetal ultrasound measurement. The patient could rest at home and be diagnosed by the expert remotely without lining up in the hospital. Meanwhile, by omitting the time to set up devices and introducing the operation to the mother in the hospital, the doctor can efficiently care for more patients to ease the shortage of medical resources and avoid the risk of the COVID-19 pandemic.

• Minimum: (Expected by Mid March, finished)

1. A web page showing real-time ultrasound images with buttons to move the ultrasound probe in a simulated system. ​
2. A conceptual mechanical design (mirror-based) of the patch ultrasound. ​
3. A report on a phantom study of an acoustic mirror.

• Expected: (Expected by late April, finished)

1. Demo the UI on a real patch ultrasound prototype.​
2. Detailed manufacturable design (CAD).

• Maximum: (Expected by date)

1. A patient end interface.​
2. A prototype to demonstrate the feasibility of mirror-based patch ultrasound.​
3. The communication between the prototype and the UI.​

User interface:

1. A web application with client and server architecture.
2. Front end: JavaScript.
3. Back end: Python.
4. Communication: Web socket.

The user interface will be a web application whose front end is realized by JavaScript, and the back end is built with Python. The front end interface interacts with a patch ultrasound on the back end to get the images in real-time and transmit the images to the interface. We plan to use the WebSocket protocol to enable interaction between the user interface and the patch ultrasound. As for remote control of the patch ultrasound, we can acquire the intended direction with buttons in the user interface. And the back-end programming moves the patch ultrasound to the desired position.

Patch Ultrasound:

1. Using a moving acoustic mirror to steer ultrasound images. ​
2. The design of patch ultrasound will enable the holdable probe to be wearable.

The patch ultrasound itself is using an acoustic mirror to reflect ultrasound waves coming from the transducer so that the operator could image the patient from multiple angles. An acoustic mirror is a part that could reflect the ultrasound wave to a specific angle without losing too much energy, as shown in Fig 1. The green section from the probe and the yellow section reflected by the mirror represents the ultrasound waves. The angle of reflected waves could be adjusted by changing the angle of the acoustic mirror M. The mirror is made from stainless steel. A detailed reason of mirror selection will be introduced in the following section. Here, with the application of an acoustic mirror, the patch ultrasound could detect the target from multiple angles without moving the probe.

Furthermore, a mechanical lock will be designed for the patch ultrasound to hold the probe and make the patch ultrasound wearable, which frees the doctor’s hands during the fetal measurement, enabling the operator to work remotely. The communication between the patch ultrasound and the user interface will be accomplished through a web socket. A block diagram shown in Figure 2 could help illustrate the technical approaches more clearly.

                          Figure 2. Technical approaches and their purpose

Milestone name: Build a basic interface to realize the image display function

• Planned Date: 3/6
• Expected Date: 3/6
• Status: Completed

Milestone name: Report on a phantom study of an acoustic mirror

• Planned Date: 3/12
• Expected Date: 3/12
• Status: Completed

Milestone name: A conceptual mechanical design

• Planned Date: 3/14
• Expected Date: 3/14
• Status: Completed

Milestone name: Realize remote control in simulation

• Planned Date: 4/3
• Expected Date: 4/3
• Status: Completed

Milestone name: A Manufacturable design

• Planned Date: 4/17
• Expected Date: 4/17
• Status: Completed

Milestone name: Build up a prototype

• Planned Date: 5/8
• Expected Date: 5/17
• Status: In progress

Milestone name: Develop a patient end interface

• Planned Date: 5/8
• Expected Date: 5/8
• Status: Not started

• Project Plan
  • Project plan presentation
  • Project plan proposal
• Project Background Reading
  • See Bibliography below for links.
  • cis_ii_critical_review
• Project Checkpoint
  • Project checkpoint presentation
• Paper Seminar Presentations
  • here provide links to all seminar presentations
  • background_reading_presentation.pdf
  • mixedrealityhumanteleoperation.pdf
  • optimizing_the_light_delivery_of_linear-array-based_photoacoustic_systems_by_double_acoustic_reflectors.pdf
  • remote_just-in-time_telementored_trauma.pdf
• Project Final Presentation
  • Poster: poster.pdf
  • Printed out prototype: printed_out_prototype.pptx
• Project Final Report
  • Final Report: final_report.pdf
  • Manufacturable Design CAD Files: manufacturable_design.zip
  • Code link for user interface: https://github.com/SrZhang4160/CIS2.git
  • User Interface Demo - Real-time Ultrasound Image Display & remote control

Reading list:

[1] B. Jiang, K. Xu, R. H. Taylor, E. Graham, M. Unberath and E. M. Boctor, “Standard Plane Extraction From 3D Ultrasound With 6-DOF Deep Reinforcement Learning Agent,” 2020 IEEE International Ultrasonics Symposium (IUS), 2020, pp. 1-4, doi: 10.1109/IUS46767.2020.9251555.

[2] Gueziri, H.-E., Santaguida, C., Collins, D.L.: The state-of-the-art in ultrasound-guided spine interventions. Medical Image Analysis 65, 101769 (2020)

[3] Hacihaliloglu, I., Rasoulian, A., Rohling, R.N., Abolmaesumi, P.: Statistical shape model to 3D ultrasound registration for spine interventions using enhanced local phase features. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 361–368 (2013). Springer

[4] Zhang, H.K., Kim, Y., Lin, M., Paredes, M., Kannan, K., Moghekar, A., Durr, N.J., Boctor, E.M.: Toward dynamic lumbar puncture guidance using needle-based single-element ultrasound imaging. Journal of Medical Imaging 5(2), 021224 (2018)

[5] Wang, Y., Lim, R.S.A., Zhang, H. et al. Optimizing the light delivery of linear-array-based photoacoustic systems by double acoustic reflectors. Sci Rep 8, 13004 (2018). https://doi.org/10.1038/s41598-018-31430-5

[6] Andrew W. Kirkpatrick, Ian McKee, Jessica L. McKee, et al. Remote just-in-time telementored trauma ultrasound: a double-factorial randomized controlled trial examining fluid detection and remote knobology control through an ultrasound graphic user interface display. The American Journal of Surgery, Volume 211, Issue 5, 2016, Pages 894-902.e1. https://doi.org/10.1016/j.amjsurg ​

[7] Black, David; Yazdi, Yas Oloumi; Hadi Hosseinabadi, Amir Hossein; Salcudean, Septimiu (2021): Human Teleoperation - A Haptically Enabled Mixed Reality System for Teleultrasound. TechRxiv. Preprint. https://doi.org/10.36227/techrxiv.15175869.v1

[1] B. Jiang, K. Xu, A. Moghekar, P. Kazanzides, and E. M. Boctor, Auto InFocus, a new paradigm for ultrasound-guided spine intervention: a multi-platform validation study. (Unpublished)