Contact Us
CiiS Lab
Johns Hopkins University
112 Hackerman Hall
3400 N. Charles Street
Baltimore, MD 21218
Directions

Lab Director
Russell Taylor
127 Hackerman Hall
rht@jhu.edu

Trace:

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
courses:456:2022:projects:456-2022-01:project-01 [2022/05/02 00:34] – [Technical Approach] mliu59courses:456:2022:projects:456-2022-01:project-01 [2022/05/02 00:43] (current) – [Other Resources and Project Files] mliu59
Line 40: Line 40:
 ======Technical Approach====== ======Technical Approach======
 For the robot homing component, we opted to develop a visual servoing algorithm for moving the robot to any arbitrary position in the cartesian robot workspace. The visual servoing procedure uses two cameras (one overhead and one at an arbitrary position) to capture two 2D projections of the robot’s tooltip and goal positions in images. The pixel positions (u, v) of the goal and current gripper tooltip can then be extracted from the images using an automated computer vision component or a manual procedure. The algorithm can then iteratively loop through motion steps to move the gripper tooltip to the target position, until the gripper tooltip and the target on both images converge, meaning that the tooltip is exactly on top of the goal position. At each of these motion steps, the robot performs a small motion in each of the axis (X, Y, and Z), finds the new tool position in the images, and determines the gradient of the tooltip’s pixel locations with respect to the small motion in each of the robot’s axis in the form of a Jacobian. This Jacobian can then be fed into a minimization algorithm to obtain a new position in robot coordinates that would minimize the distance between the goal position’s image projection pixel coordinate and the new robot position’s image projection pixel coordinate. For the robot homing component, we opted to develop a visual servoing algorithm for moving the robot to any arbitrary position in the cartesian robot workspace. The visual servoing procedure uses two cameras (one overhead and one at an arbitrary position) to capture two 2D projections of the robot’s tooltip and goal positions in images. The pixel positions (u, v) of the goal and current gripper tooltip can then be extracted from the images using an automated computer vision component or a manual procedure. The algorithm can then iteratively loop through motion steps to move the gripper tooltip to the target position, until the gripper tooltip and the target on both images converge, meaning that the tooltip is exactly on top of the goal position. At each of these motion steps, the robot performs a small motion in each of the axis (X, Y, and Z), finds the new tool position in the images, and determines the gradient of the tooltip’s pixel locations with respect to the small motion in each of the robot’s axis in the form of a Jacobian. This Jacobian can then be fed into a minimization algorithm to obtain a new position in robot coordinates that would minimize the distance between the goal position’s image projection pixel coordinate and the new robot position’s image projection pixel coordinate.
 +{{ :courses:456:2022:projects:456-2022-01:piecture5.png?400 |}}
 +The robot homing procedure in two cameras
 +
 +
 +For the robot tool path calibration component, the goal is to develop a more intuitive, human understandable, and unified data structure for the representation of waypoints used in the robot’s normal operation. In order to achieve this, we developed a custom data structure that does not explicitly define each robot waypoint as its own set of absolute coordinates, but rather as a “directional graph” of positions relative to each other and one single absolute “home position”. This calibration graph is stored in a human readable JSON file for ease of calibration, and is loaded by a ROS node upon robot system startup.
 +{{ :courses:456:2022:projects:456-2022-01:picture2.png?600 |}}
 +The graph developed for the current robot's operational trajectory. Each node is a specific waypoint along some trajectory. 
  
-For the robot toll path calibration component, the goal is to develop a more intuitive, human understandable, and unified data structure for the representation of waypoints used in the robot’s normal operation. In order to achieve this, we developed a custom data structure that does not explicitly define each robot waypoint as its own set of absolute coordinates, but rather as a “directional graph” of positions relative to each other and one single absolute “home position”. This calibration graph is stored in a human readable JSON file for ease of calibration, and is loaded by a ROS node upon robot system startup. 
  
 For the robot's handeye calibration, since there is already a calibration procedure that can handle the bulk of the calibration, our team would package the script such that it can be integrated into the ROS framework for the robot. This existing procedure moves the robot to N points in a rectangular grid near the 2D plane where the mosquito manipulation occurs, detects the robot gripper tooltip position at each of these points, and fits the resulting point correspondences using a Bernstein polynomial fit. Coefficients for this polynomial fit is then saved in a config file. This config file is then opened and read by a dedicated ROS service node that can translate between robot and camera coordinates.  For the robot's handeye calibration, since there is already a calibration procedure that can handle the bulk of the calibration, our team would package the script such that it can be integrated into the ROS framework for the robot. This existing procedure moves the robot to N points in a rectangular grid near the 2D plane where the mosquito manipulation occurs, detects the robot gripper tooltip position at each of these points, and fits the resulting point correspondences using a Bernstein polynomial fit. Coefficients for this polynomial fit is then saved in a config file. This config file is then opened and read by a dedicated ROS service node that can translate between robot and camera coordinates. 
  
-All three of the components above are to be implemented as each its own ROS node, along with a calibration GUI that allows for direct supervision of any calibration process by the operator.+All three of the components above are to be implemented as each its own ROS node, along with a calibration GUI that allows for direct supervision of any calibration process by the operator. Below are GUIs developed in rqt for the robot homing and handeye calibration components. 
 +{{ :courses:456:2022:projects:456-2022-01:picture3.png?600 |}} 
 +Screenshot of the handeye calibration GUI 
 + 
 +{{ :courses:456:2022:projects:456-2022-01:picture3.png?600 |}} 
 +Screenshot of the robot homing GUI
 ======Dependencies====== ======Dependencies======
 Significant dependencies: Significant dependencies:
Line 67: Line 78:
     * Planned Date: 03/31/2022     * Planned Date: 03/31/2022
     * Expected Date: 03/31/2022     * Expected Date: 03/31/2022
-    * Status: In Progress+    * Status: Complete
   - Milestone name:  System integration and verification testing complete   - Milestone name:  System integration and verification testing complete
     * Planned Date: 04/20/2022     * Planned Date: 04/20/2022
     * Expected Date: 04/20/2022     * Expected Date: 04/20/2022
-    * Status: In Progress+    * Status: Partially complete
   - Milestone name:  Finalize system documentation + Final project presentation materials   - Milestone name:  Finalize system documentation + Final project presentation materials
     * Planned Date: 05/04/2022     * Planned Date: 05/04/2022
     * Expected Date: 05/04/2022     * Expected Date: 05/04/2022
-    * Status: Not Started+    * Status: Complete
  
 ======Reports and presentations====== ======Reports and presentations======
Line 91: Line 102:
     * {{:courses:456:2022:projects:456-2022-01:final_poster_pdf.pdf|PDF of Poster}}     * {{:courses:456:2022:projects:456-2022-01:final_poster_pdf.pdf|PDF of Poster}}
   * Project Final Report   * Project Final Report
-    * {{:courses:456:2022:projects:456-2022-01:final_report.pdf|Final Report}}+    * {{ :courses:456:2022:projects:456-2022-01:liu_cis_ii_final_report.pdf |}}
     * links to any appendices or other material     * links to any appendices or other material
  
Line 99: Line 110:
 W. Li et al., "Automated Mosquito Salivary Gland Extractor for PfSPZ-based Malaria Vaccine Production," 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 866-872, doi: 10.1109/ICRA48506.2021.9560959. W. Li et al., "Automated Mosquito Salivary Gland Extractor for PfSPZ-based Malaria Vaccine Production," 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 866-872, doi: 10.1109/ICRA48506.2021.9560959.
 ======Other Resources and Project Files====== ======Other Resources and Project Files======
-Here give list of other project files (e.g., source code) associated with the project.  If these are online give a link to an appropriate external repository or to uploaded media files under this name space (2022-01).+Code repositories: 
 +All of the code for the robot, including the code developed as a part of this project, are stored in this repository group on GitLab:  
 +https://git.lcsr.jhu.edu/mosquitoproject 
 + 
 +The code repositories directly containing work covered by this project are: 
 + 
 +https://git.lcsr.jhu.edu/mosquitoproject/rqt_sanaria_robot_controller 
 +https://git.lcsr.jhu.edu/mosquitoproject/sanaria_calibration_tools 
 +https://git.lcsr.jhu.edu/mosquitoproject/sanaria_transformation_graph 
 +https://git.lcsr.jhu.edu/mosquitoproject/sanaria_handeye 
 + 
 +Documentation on system design is in the documentation repository:  
 +https://git.lcsr.jhu.edu/mosquitoproject/sanaria_docs 
courses/456/2022/projects/456-2022-01/project-01.1651451664.txt.gz · Last modified: by mliu59




ERC CISST    LCSR    WSE    JHU