Last updated: 04/20/2022

We would like to propose a solution that uses an RL agent to make input decisions for PCVs(Pressure Controlled Ventilatior).

• Students: Haoyu Shi, Jiahe Xu
• Mentor(s): Antwan D. Clark

You may want to include a picture or two here.

Pressure Controlled Ventilation has several points that are crucial to being implemented correctly. The parameters must be set to achieve a certain tidal volume. Given that this is not one of the direct inputs, it will require calculations. Extra attention must also be given to the max pressure value set to avoid barotrauma.

• Minimum: (Expected by 4/6)
  1. build a reliable mathematical lung model and a PCV model
  2. train a RL agent that automatically makes decisions to adjust the parameters

• Expected: (Expected by 4/14)
  1. initial proof of concepts of MDP in ventilator controller
  2. documented code and performance metrics.

• Maximum: (Expected by 4/28)
  1. Conference paper

With mathematical models of lung(multi-componential models), we could have an reliable simulation for lung reaction. (RL environment) Change pressure control ventilator settings(RL action) to adjust the state of lung (Flow, Pressure, Volume, etc : RL agent state) Use MIMIC-III dataset and Survival Analysis to evaluate the cost/reward of RL actions

• MIMIC III Impact: on Training Results and Evaluations (Completed)
• Computing Resources Impact: Training the Network and Accessing the Database
• Interface Between Python and Matlab Simulink : A larger workload for migrating Model to Python (Completed)

1. Milestone name: Regular PCV model with Lung
   • Planned Date: 2/18/2022
   • Expected Date: 2/18/2022
   • Status: Completed
2. Milestone name: Explore and define legit actions for RL agent(PCV controller)
   • Planned Date: 3/14/2022
   • Expected Date: 3/14/2022
   • Status: Completed
3. Milestone name: Build and Evaluate MDP
   • Planned Date: 3/21/2022
   • Expected Date: 3/21/2022
   • Status: Completed
4. Milestone name: Build the RL environment
   • Planned Date: 3/25/2022
   • Expected Date: 3/25/2022
   • Status: Completed
5. Milestone name: Exploration of RL networks and methods
   • Planned Date: 4/6/2022
   • Expected Date: 4/6/2022
   • Status: In Progress
6. Milestone name: Train a RL agent and find metrics to evaluate its performance
   • Planned Date: 4/14/2022
   • Expected Date: xxx
   • Status: In Progress
7. Milestone name: Evaluate and Improve Model
   • Planned Date: 4/28/2022
   • Expected Date: xxx
   • Status: Not Started

• Project Plan
  • Project Proposal Presentation
  • Project Plan Proposal
• Project Background Reading
  • See Bibliography below for links.
• Project Checkpoint
  • Checkpoint Presentation
• Paper Seminar Presentations(Original Paper in Other Resources Section)
  • Background Report
  • Background Presentation
• Project Final Presentation
  • Final Poster
• Project Final Report
  • Final Report
  • links to any appendices or other material

• Messina, Z. (2021, August 2). Pressure controlled ventilation. StatPearls [Internet]. Retrieved February 22,

2022, from https://www.ncbi.nlm.nih.gov/books/NBK555897/

• Mohammad Jaber, Lara Hamawy and Mohamad Hajj-Hassan et al. MATLAB/Simulink Mathematical Model for Lung and

Ventilator. DOI: 10.1109/ICM50269.2020.9331820

• H. Y. Al-Hetari, Y. Alginahi, M. N. Kabir, N. Q. Al-Naggar, M. A. Al-Rumaima and M. M. Hasan, “Modeling Lung Functionality in Volume-Controlled Ventilation for Critical Care Patients,” 2020 IEEE 2nd International Conference on Artificial Intelligence in Engineering and Technology (IICAIET), 2020, pp. 1-4, doi: 10.1109/IICAIET49801.2020.9257851.

• Al Naggar, N.Q. (2015). Modelling and Simulation of Pressure Controlled Mechanical Ventilation System. Journal of Biomedical Science and Engineering, 08, 707-716.

• Yan Shi, Shuai Ren, Maolin Cai, Weiqing Xu, Qiyou Deng, “Pressure Dynamic Characteristics of Pressure Controlled Ventilation System of a Lung Simulator”, Computational and Mathematical Methods in Medicine, vol. 2014, Article ID 761712, 10 pages, 2014. https://doi.org/10.1155/2014/761712

• Initial Timeline
• Updated Timeline
• Modelling and Simulation of Pressure Controlled Mechanical Ventilation System