Last updated: 5/04/2015 10:00 PM

Photoacoustic (PA) imaging currently requires specialized hardware to capture and process PA signals. Though conventional ultrasound (US) systems can also capture PA signals, such systems beamform the PA signals incorrectly, resulting in distorted PA images. By developing and applying a real-time PA rebeamforming algorithm in an ultrasound platform, we hope to demonstrate a cheaper and more flexible method for PA beamforming.

• Students: Howard Huang
• Mentor(s): Emad Boctor, Haichong “Kai” Zhang

/ You may want to include a picture or two here./ Here are the US Interfaces (Porta, Ulterius) built using Ultrasonix SDK. We intend to build an image display and set up the PA imaging option into the Ulterius interface by the end of the semester.

Original Ulterius Interface and Updated Interface with Image Display

A paragraph or so here. Give background of the problem. Explicitly state specific aims (numbered list is good) and explain why they are important

Photoacoustic (PA) imaging is a medical imaging modality whose uses include blood vessel visualization, cancer detection, and tracking of surgical tools in the body (e.g. brachytherapy seeds in prostate). Current PA imaging systems require specialized hardware to process PA signals. Even though conventional ultrasound (US) systems can receive PA signals, they beamform the data incorrectly as pulse-echo signals, resulting in distorted PA images. This limitation raises the overall cost of PA imaging and reduces its availability. By incorporating PA imaging into existing clinical US systems, this project hopes to reduce PA imaging costs and increase its usage in clinical settings.

Thus our specific aims are to:

1. Develop a fast rebeamforming algorithm for PA signals on US systems.
2. Integrate algorithm into a real-time US visualization program (Ultrasonix SDK: Ulterius).
3. Set up functional real-time PA imaging on a clinical US platform (use the US platform to capture real-time PA signals, and process signal into images using Ulterius).

As stated previously, the significance of integrating PA imaging into an US platform is that it significantly reduces the cost/hardware requirements of PA imaging, hopefully making it a more flexible and cost-efficient imaging modality.

• Minimum: (Expected by Mid-March (3/16)) - All Minimum Deliverables Completed
  1. Documentation of PA re-beamforming algorithm and its integration into an US visualization platform.
  2. Convert existing re-beamforming algorithm to C++.
  3. Create scripts to debug algorithm with simulation data sets (basic results).

• Expected: (Expected by Mid-April (4/15)) - All Expected Deliverables Completed
  1. Adapt existing US platform to allow for PA imaging. Integrate our PA beamformer into system.
  2. Construct PA/US phantoms. Setup experiments to test PA imaging system.
  3. Test PA imaging system using real RF/US data (collect more detailed results).

• Maximum: (Expected by Late-April (4/30)) - Demo and Paper Draft Completed
  1. Summarize current findings in a paper for submission.
  2. Present a live demo of real time PA imaging system.
  3. Integrate alternative PA image algorithms (inverse beamforming, US visual data) into completed PA system.

The third deliverable has always been the least important deliverable for our system, as the other two algorithms perform the same task as our PA rebeamformer, but are slower or less accurate (they are redundant components for the real-time imaging system). We would like to spend the remaining time working on the other maximum deliverables as well as refining our system for use by other interested researchers.

here describe the technical approach in sufficient detail so someone can understand what you are trying to do

Since the project goal is to develop a real time PA imaging system, my priority is to first build the PA re-beamforming algorithm in C++ and then integrate the algorithm into an existing ultrasound interface ( Ultrasonix SDK) to allow for fast PA imaging.

Available US interfaces from the Ultrasonix SDK include Porta and Ulterius. Ulterius is an US interface that allows for access to the US machine over a network connection. This means we can run and collect results from the US machine remotely using a PC or laptop (which is very convenient for code development and testing).

The current Ulterius interface only displays options for data collection (no image display is available). However our lab does have access to an Ulterius build with image display (using additional openCV, QT modifications) that we will be establishing in our own build.

As a result our current technical approach is to:

1. Develop the rebeamforming algorithm in C++ (Documentation included in Project Files section)
2. Build a functional Ultrasonix Ulterius interface (default build from SDK)
3. Implement Ulterius US image visualization (use reference SDK from lab, documentation included in Project Files)
4. Modify implementation for PA image visualization

describe dependencies and effect on milestones and deliverables if not met:

1. PA re-beamforming algorithm (Acquired from Mentor)
2. US Ultrasonix SDK Software (Acquired)
3. Ultrasonix Ulterius build with US image visualization (Acquired)
4. US System and Probe (Present)
5. Access to Robotorium and lab (Acquired)
6. PA inverse beamforming and video processing algorithm (Available)
7. PA Image Setup (Laser system, PZT element as source) (Available for setup)
8. US Phantom (Basic phantoms Available)

Our first dependency is the US-to-PA rebeamforming algorithm. Without this dependency, our primary milestone (implementation of rebeamforming algorithm in C++) would not be possible.

Other major dependencies are access to lab and US system (physical system + Ultrasonix SDK), which are needed for us to integrate the PA imaging algorithm into the US workflow.

All major dependencies have been acquired or are available for use (ex. PA imaging system and US system are present in lab).

Current Milestones in Progress:

1. Milestone name: Develop C++ Rebeamforming Algorithm
   • Planned Date: 03/16
   • Expected Date: 03/16
   • Status: Code Developed (Awaiting integration into Ultrasonix) Might need more focus on. Will wrap up in place of third milestone by this weekend.
2. Milestone name: Establish Functional Ultrasonix Build
   • Planned Date: 03/23
   • Expected Date: 03/23
   • Status: Completed.
3. NEW Milestone: Develop Visualization on Ulterius Interface
   • Planned Date: 03/23
   • Expected Date: 03/30
   • Status: Completed (way ahead of schedule!!! :D)
4. Milestone Name: Implement Rebeamforming Algorithm on Ulterius Interface
   • Planned Date: 04/10
   • Expected Date: 04/10
   • Status: Completed! :)
5. Milestone Name: Implement PA Imaging using Rebeamformed Data on Ulterius Interface
   • Planned Date: 04/17
   • Expected Date: 04/20
   • Status: Completed! :)
6. Milestone Name: Use Realtime System to Collect PA Image Data
   • Planned Date: 04/30
   • Expected Date: 04/30
   • Status: Completed! (Although we are using pseudo-PA signals from PZT elements).
7. Milestone Name: Refine Ulterius Interface (Re-scale Image to match US positions)
   • Planned Date: 04/25
   • Expected Date: 04/25
   • Status: Completed
8. Max Deliverable Milestone: Draft Results and Documentation into a User Manuscript/Paper
   • Planned Date: 05/05 (Will need to show mentors by 05/04)
   • Expected Date: 05/05
   • Status: Completed
9. Max Deliverable Milestone: Prepare Demo for Class Presentation.
   • Planned Date: 05/06
   • Expected Date: 05/06
   • Status: In Progress (Will need access to PZT element, Ultrasonix Touch machine, Basic PA Phantom Chamber, and PZT power supply on day of demo).

• Project Plan
  • presentation13_final.pdf| Project plan presentation}}
  • group_13_project_proposal_final.pdf|Project plan proposal}}
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • presentation13_checkpoint_final.pdf
• Paper Seminar Presentations
  • here provide links to all seminar presentations
  • Seminar Presentation: seminar_presentation_final.pdf
  • Seminar Report: group_13_seminar_final.pdf
• Project Final Presentation
  • poster_howard_huang.pdf
  • project13teaser.pptx
• Project Final Report: project_13_report.pdf
• Video Demo: ulterius_demo_5_2_2016_6_30_34_pm_2_.zip
  • links to any appendices or other material

* here list references and reading material

Zhang, Kai, et. al. “Synthetic Aperture Based Photoacoustic Image Re-beamforming From Ultrasound Post-beamformed RF Data”. Unpublished Manuscript (will be submitted for publication).

Park, Suhyun, et al. “Adaptive beamforming for photoacoustic imaging using linear array transducer.” Ultrasonics Symposium, 2008. IUS 2008. IEEE. IEEE, 2008.

Kuo, Nathanael, et al. “Real-time photoacoustic imaging of prostate brachytherapy seeds using a clinical ultrasound system.” Journal of biomedical optics 17.6 (2012): 0660051-0660057.

Kang, Hyun-Jae, et al. “Software framework of a real-time pre-beamformed RF data acquisition of an ultrasound research scanner.” SPIE Medical Imaging. International Society for Optics and Photonics, 2012.

Harrison, Travis, and Roger J. Zemp. “The applicability of ultrasound dynamic receive beamformers to photoacoustic imaging.” Ultrasonics, Ferroelectrics, and Frequency Control, IEEE Transactions on 58.10 (2011): 2259-2263.

Frazier, Catherine H., and William Brien. “Synthetic aperture techniques with a virtual source element.” Ultrasonics, Ferroelectrics, and Frequency Control, IEEE Transactions on 45.1 (1998): 196-207.

J. Kortbek, J. A. Jensen, K. L. Gammelmark, “Synthetic Aperture Sequential Beamforming,” Proc. in IEEE Int. Ultrasonics Symp., 966-969 (2008).

Wilson, Thaddeus, et al. “The ultrasonix 500RP: A commercial ultrasound research interface.” Ultrasonics, Ferroelectrics, and Frequency Control, IEEE Transactions on 53.10 (2006): 1772-1782.

/*Here give list of other project files (e.g., source code) associated with the project. If these are online give a link to an appropriate external repository or to uploaded media files under this name space.*/

Overall Documentation: overall_documentation_demos_ulterius_interface_.pdf

Live Ulterius Demo: ulterius_release.zip

Simulation PA Demo:pa_simulation_code.zip

Real Data PA Demo: pa_real_data_analysis_code.zip

Current Ultrasonix Ulterius Source Code: sdk607_final_version.zip

Rebeamforming Algorithm Implementation: Rebeamforming C++ Pseudocode:image_13_.jpg

Function parameters: double* pa_rebeamforming(const int ns, const int nl, double channelSpacing, double speedSound, double* rf) Function input description:

1. rf = array of RF data (in double or int_16 format)
2. ns,nl = dimensions of RF data( # samples per line, # lines)
3. channelSpacing = distance between lines
4. speedSound = speed of sound (for depth calculations)

This algorithm calculates a new time delay (based on channelSpacing and speedSound parameters) and then applies another round of dynamic beamforming on data. The beamforming effectively sums together elements in the array based on this time delay (where depth is roughly 1/2 that of the original beam-forming).

Output = rebeamformed rf array (of same dimension as input array)

Ultrasonix Ulterius Documentation: Ulterius source code is provided within the Ultrasonix SDK package(sdk607)

To build Ulerius code on Windows:

1. Download Microsoft Visual Studios (2010 version suggested by Ultrasonix documentation, but difficult to find).
2. Download CMake 2.8.8, QT4 (4.8.7, use Visual Studios 2010 add-in), and opencv(2.4.11).
3. Run CMake 2.8.8 on sdk607 directory (create a new directory for project files)
4. Establish QT4 and opencv libraries in Visual Studios 2010.

/Note: Some of these programs (esp. Visual Studios 2010) are difficult to find. Updating the code to use newer dependencies may be a possible max deliverable or something I can refine after completing the main deliverables./

Implementation of Image Panel on Ulterius Interface: (Many thanks to Lei for providing a reference Ulterius build with QT visualization)

• Modified Ulterius.ui file. Added in a viewing plane object using QT software interface on VS 2010.

In UlteriusDemo.cpp:

• Modified onNewData function (recieves data as an array from the system) to transfer array to processFrame function.

• Added processFrame function - A new function built on the UlteriusDemo.cpp. As inputs: it takes in data (1D array of chars/ints) and data type (int). Data is then transformed into a 2d array of pixels (QTarray) and send into the viewing plane object.
• Adapted PA rebeamformer into Ulterius to allow for real-time PA imaging. Made modifications to allow for image averaging and scan conversion as well.

Effectively, the workflow for the new Ulterius interface is: Data from system –> calls onNewData function –> calls processFrame function –> display image.

By adding our rebeamforming function into onNewData function (effectively convert RF array data to B mode array data) we can then send the data and proper image type into the processFrame function to display a a properly beamformed image for PA signals. We can also apply envelope detection and log compression to refine the image further.