Trace:

Differences

This shows you the differences between two versions of the page.

--- courses:456:2022:projects:456-2022-01:project-01 [2022/03/14 04:53] – [Vision-Based Automatic Calibration of Mosquito Dissection System for the Production of Malaria Vaccine] mliu59
+++ courses:456:2022:projects:456-2022-01:project-01 [2022/05/02 00:43] (current) – [Other Resources and Project Files] mliu59
@@ Line 1: / Line 1: @@
 ======Vision-Based Automatic Calibration of Mosquito Dissection System for the Production of Malaria Vaccine======
-**Last updated: 03/14/2022 11:00PM**
+**Last updated: 05/01/2022 08:00PM**
@@ Line 26: / Line 26: @@
     - Prototype implementation of new “calibration tree” **Complete**
     - Significant progress with development of robot homing calibration procedures **Complete**
-    - Documentation
+    - Documentation **Complete**
   * **Expected:** (Expected by Mid April)
-    - A single calibration workflow & GUI that covers all calibration tasks
+    - A single calibration workflow & GUI that covers all calibration tasks **Partially Complete**
-    - Deployment version of the “calibration tree”
+    - Deployment version of the “calibration tree” **Partially Complete**
   * **Maximum:** (Expected by Early May)
@@ Line 39: / Line 39: @@
 ======Technical Approach======
-For the robot homing component, our team is planning on tackling this with a vision-based approach, where we would use the fixture's existing cameras to guide to robot to a fixed home position using visual-servoing.
+For the robot homing component, we opted to develop a visual servoing algorithm for moving the robot to any arbitrary position in the cartesian robot workspace. The visual servoing procedure uses two cameras (one overhead and one at an arbitrary position) to capture two 2D projections of the robot’s tooltip and goal positions in images. The pixel positions (u, v) of the goal and current gripper tooltip can then be extracted from the images using an automated computer vision component or a manual procedure. The algorithm can then iteratively loop through motion steps to move the gripper tooltip to the target position, until the gripper tooltip and the target on both images converge, meaning that the tooltip is exactly on top of the goal position. At each of these motion steps, the robot performs a small motion in each of the axis (X, Y, and Z), finds the new tool position in the images, and determines the gradient of the tooltip’s pixel locations with respect to the small motion in each of the robot’s axis in the form of a Jacobian. This Jacobian can then be fed into a minimization algorithm to obtain a new position in robot coordinates that would minimize the distance between the goal position’s image projection pixel coordinate and the new robot position’s image projection pixel coordinate.
+{{ :courses:456:2022:projects:456-2022-01:piecture5.png?400 |}}
+The robot homing procedure in two cameras
-For the robot toll path calibration component, our team is planning on building a custom "calibration tree" data structure to store robot waypoint configurations. This data structure would define a set of hardware-defined waypoint coordinates relative to the absolute robot home position and each other. The robot can then access waypoint coordinates by parsing this data structure.
-For the robot's handeye calibration, since there is already a calibration procedure that can handle the bulk of the calibration, our team would package the script such that it can be integrated into the ROS framework for the robot.
+For the robot tool path calibration component, the goal is to develop a more intuitive, human understandable, and unified data structure for the representation of waypoints used in the robot’s normal operation. In order to achieve this, we developed a custom data structure that does not explicitly define each robot waypoint as its own set of absolute coordinates, but rather as a “directional graph” of positions relative to each other and one single absolute “home position”. This calibration graph is stored in a human readable JSON file for ease of calibration, and is loaded by a ROS node upon robot system startup.
+{{ :courses:456:2022:projects:456-2022-01:picture2.png?600 |}}
+The graph developed for the current robot's operational trajectory. Each node is a specific waypoint along some trajectory.
-Ultimately, in order to develop a full calibration procedure with all subcomponents mentioned above, we would develop a operator-centered GUI that can provide the supervising operator manual control over the procedure as well as pertinent calibration information.
+For the robot's handeye calibration, since there is already a calibration procedure that can handle the bulk of the calibration, our team would package the script such that it can be integrated into the ROS framework for the robot. This existing procedure moves the robot to N points in a rectangular grid near the 2D plane where the mosquito manipulation occurs, detects the robot gripper tooltip position at each of these points, and fits the resulting point correspondences using a Bernstein polynomial fit. Coefficients for this polynomial fit is then saved in a config file. This config file is then opened and read by a dedicated ROS service node that can translate between robot and camera coordinates.
+All three of the components above are to be implemented as each its own ROS node, along with a calibration GUI that allows for direct supervision of any calibration process by the operator. Below are GUIs developed in rqt for the robot homing and handeye calibration components.
+{{ :courses:456:2022:projects:456-2022-01:picture3.png?600 |}}
+Screenshot of the handeye calibration GUI
+{{ :courses:456:2022:projects:456-2022-01:picture3.png?600 |}}
+Screenshot of the robot homing GUI
 ======Dependencies======
 Significant dependencies:
   - Availability of work station, robot core components such as cameras, robot arm, etc.
       * While a significant amount of work can be done online, not having the physical hardware available would hinder the project's progress for further system integration and validation. It will also prevent the team from being able to develop a proper vision-based calibration algorithm.
+      * **Resolved** Scheduling calendar and robot usage logs
   - Frequency of hardware changes
       * If during the testing and validation process, other teams implement frequent hardware changes, it may affect our ability to collect meaningful data.
-  - Availability of support on Computer Vision tasks
+      * **Resolved** Robot usage logs and frequent communication with lab team
-      * If there is not availability of vision support, some component will not be able to be implemented through a computer vision approach, and would require a hardware solution.
+  - Tooltip detection algorithm **Only affects maximum deliverables**
+      * A tooltip detection algorithm is critical to fully automating vision based calibration procedures, but there are workarounds to manually label tooltips.
+  - Robot controller PID tuning **Only affects maximum deliverables**
+      * Affects the accuracy achievable with calibration. May or may not be critical to fully automated calibration, further investigation is needed.
 ======Milestones and Status ======
@@ Line 63: / Line 78: @@
     * Planned Date: 03/31/2022
     * Expected Date: 03/31/2022
-    * Status: In Progress
+    * Status: Complete
   - Milestone name:  System integration and verification testing complete
     * Planned Date: 04/20/2022
     * Expected Date: 04/20/2022
-    * Status: Not Started
+    * Status: Partially complete
   - Milestone name:  Finalize system documentation + Final project presentation materials
     * Planned Date: 05/04/2022
     * Expected Date: 05/04/2022
-    * Status: Not Started
+    * Status: Complete
 ======Reports and presentations======
@@ Line 81: / Line 96: @@
     * {{ :courses:456:2022:projects:456-2022-01:liu_cis_ii_background_reading_critical_review.pdf |}}
   * Project Checkpoint
-    * {{:courses:456:2022:projects:456-2022-01:checkpoint_presentation.pdf| Project checkpoint presentation}}
+    * {{ :courses:456:2022:projects:456-2022-01:liu_cis_ii_checkpoint_presentation.pdf |}}
   * Paper Seminar Presentations
     * {{ :courses:456:2022:projects:456-2022-01:liu_cis_ii_background_presentation_slides.pdf |}}
@@ Line 87: / Line 102: @@
     * {{:courses:456:2022:projects:456-2022-01:final_poster_pdf.pdf|PDF of Poster}}
   * Project Final Report
-    * {{:courses:456:2022:projects:456-2022-01:final_report.pdf|Final Report}}
+    * {{ :courses:456:2022:projects:456-2022-01:liu_cis_ii_final_report.pdf |}}
     * links to any appendices or other material
@@ Line 95: / Line 110: @@
 W. Li et al., "Automated Mosquito Salivary Gland Extractor for PfSPZ-based Malaria Vaccine Production," 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 866-872, doi: 10.1109/ICRA48506.2021.9560959.
 ======Other Resources and Project Files======
-Here give list of other project files (e.g., source code) associated with the project.  If these are online give a link to an appropriate external repository or to uploaded media files under this name space (2022-01).
+Code repositories:
+All of the code for the robot, including the code developed as a part of this project, are stored in this repository group on GitLab:
+https://git.lcsr.jhu.edu/mosquitoproject
+The code repositories directly containing work covered by this project are:
+https://git.lcsr.jhu.edu/mosquitoproject/rqt_sanaria_robot_controller
+https://git.lcsr.jhu.edu/mosquitoproject/sanaria_calibration_tools
+https://git.lcsr.jhu.edu/mosquitoproject/sanaria_transformation_graph
+https://git.lcsr.jhu.edu/mosquitoproject/sanaria_handeye
+Documentation on system design is in the documentation repository:
+https://git.lcsr.jhu.edu/mosquitoproject/sanaria_docs

courses/456/2022/projects/456-2022-01/project-01.1647233631.txt.gz · Last modified: 2022/03/14 04:53 by mliu59