Trace:

Differences

This shows you the differences between two versions of the page.

--- courses:456:2022:projects:456-2022-10:project-10 [2022/04/30 19:21] – [Reports and presentations] achen134
+++ courses:456:2022:projects:456-2022-10:project-10 [2022/05/16 20:18] (current) – [Reports and presentations] achen134
@@ Line 7: / Line 7: @@
   * **Students:** An Chi Chen
-  * **Mentor(s):** Seth Billings, Chi Ewulum
+  * **Mentor(s):** Dr Seth Billings, Chigozie Ewulum
@@ Line 45: / Line 45: @@
   * **Adjusted Maximum:** (Completed on 30 April 2022)
     - Integrated haptic and auditory feedback system with simulated Argus II feedback on virtual reality headset
+======Dependencies======
+There are not many dependencies for this project however they will affect the deliverables if they are not resolved in a timely manner.  For both the haptic and auditory feedback systems the SLAM-based navigational system is required for full integration with the Argus II system.  This also includes the necessary hardware such as cameras.  The SLAM system itself has been developed and is currently in the refinement phase.  The required cameras for the system have been obtained.  The only outstanding hardware are the camera mounts – these are currently being 3D printed.  If these mounts are not completed by 10 April 2022, then the subsequent tasks (testing of gaze and path guidance and object localisation in the real world and participant testing) will be delayed.  If the delay is substantial (past 17 April 2022) it may not be possible to complete the participant testing in this timeframe as there should be at least two weeks of participant testing.
+For the participant testing, the IRB training will need to be completed.  Whether testing can take place also depends on participant availability.  The IRB training can be completed in advance and thus will be done so.  A testing protocol currently exists, and the team members will simply need to be added to it to be able to perform participant testing.  Furthermore, four weeks have been allocated to acquire participants.  Ideally the testing will be done with Argus II users, however if they are not available it is possible to perform participant testing with seeing individuals.  The system developed by APL does also connects to a VR headset which allows seeing individuals to see what Argus II users perceive.  If it is still not possible to acquire the participants by 17 April 2022 then the allocated time for participant testing will be reduced and potentially may not be completed in this project timeline.
+The table below summarises the dependencies mentioned above.
+{{ :courses:456:2022:projects:456-2022-10:dependencies.png?650 |}}
+*Latest Date = If dependency not resolved by this date it will affect the deliverables
+The following table shows the final status of the dependencies at the end of the project timeline (1 May 2022).
+{{ :courses:456:2022:projects:456-2022-10:dependenciesatend.png?650 |}}
+As can be seen the IRB and participant availability dependency were not achieved.  This was due to the fact that the existing IRB protocol only includes recruiting participants from APL or Argus II users.  The developed supplementary feedback systems, though integrated, are not in a complete enough state to be tested with Argus II users at this stage.  In addition, there was not enough time during this project timeline to execute the fallback plan for participant testing which was to test the developed integrated system with APL employees.  As seen above under the deliverables, the maximum deliverable was adjusted due to these time constraints and IRB dependency not being met.
 ======Technical Approach======
@@ Line 78: / Line 96: @@
 {{ :courses:456:2022:projects:456-2022-10:visualfeedbackimage.png?400 |}}
-======Dependencies======
-There are not many dependencies for this project however they will affect the deliverables if they are not resolved in a timely manner.  For both the haptic and auditory feedback systems the SLAM-based navigational system is required for full integration with the Argus II system.  This also includes the necessary hardware such as cameras.  The SLAM system itself has been developed and is currently in the refinement phase.  The required cameras for the system have been obtained.  The only outstanding hardware are the camera mounts – these are currently being 3D printed.  If these mounts are not completed by 10 April 2022, then the subsequent tasks (testing of gaze and path guidance and object localisation in the real world and participant testing) will be delayed.  If the delay is substantial (past 17 April 2022) it may not be possible to complete the participant testing in this timeframe as there should be at least two weeks of participant testing.
-For the participant testing, the IRB training will need to be completed.  Whether testing can take place also depends on participant availability.  The IRB training can be completed in advance and thus will be done so.  A testing protocol currently exists, and the team members will simply need to be added to it to be able to perform participant testing.  Furthermore, four weeks have been allocated to acquire participants.  Ideally the testing will be done with Argus II users, however if they are not available it is possible to perform participant testing with seeing individuals.  The system developed by APL does also connects to a VR headset which allows seeing individuals to see what Argus II users perceive.  If it is still not possible to acquire the participants by 17 April 2022 then the allocated time for participant testing will be reduced and potentially may not be completed in this project timeline.
+======Results======
-The table below summarises the dependencies mentioned above.
-{{ :courses:456:2022:projects:456-2022-10:dependencies.png?650 |}}
+**Haptic Feedback System**
-*Latest Date = If dependency not resolved by this date it will affect the deliverables
+Through the iterative testing it was found that static patterns were optimal for relaying the left and right instructions.  This was sufficient and intuitive enough to indicate to users in which direction they should look.  All 4 of the actuators located around the temporal and back of the head were utilised for this group of instructions thus making it possible to indicate a direction located towards the back of the head.
-The following table shows the final status of the dependencies at the end of the project timeline (1 May 2022).
+Dynamic patterns were employed to relay the up and down instructions.  Interestingly, when only the 3 actuators located along the top of the head were used in the dynamic patterns the interpretation of the haptic pattern became unclear – potentially due to the lower density of actuators along the top of the head.  Thus, these patterns make use of all 7 actuators actuated in a specific order.  By using all 7 actuators, the feedback from the haptic pattern felt more distinct and clearer in relaying a directional instruction.  The up pattern starts with the actuators closest to the front of the head and moves towards the back, whilst the down pattern starts at the back of the head and ends in the front.
-{{ :courses:456:2022:projects:456-2022-10:dependenciesatend.png?650 |}}
+{{ :courses:456:2022:projects:456-2022-10:hapticpatterns.png?600 |}}
+During the testing, it was realised that the duration of a haptic pattern should be as short as possible to allow for a fast update time.  If this was not satisfied, there was a noticeable delay between a turning of the head and the next correct haptic instruction which results in an overshoot of head turning movement and creates confusion.  Thus, for the static patterns the durations were set to 0.015 seconds.  This may seem like a short duration but the haptic patterns are repeated continuously until the target gaze direction is reached.  For the dynamic patterns, it is important that the durations are still set at a suitable time which makes them discernible as the directional instruction which they are relaying.  A total duration of 0.18 seconds was found to be a sufficient duration that satisfies these criteria.
+Furthermore, it was found that it is important that the haptic actuators are flush and are placed with a bit of pressure against the head in order to localise them properly.  Additionally, hair density may result in a reduced ability to determine which haptic actuator is on.  This potential issue could be solved by ensuring the haptic actuators are very close to the surface of the head.
+**Auditory Feedback System**
+A tournament to find a best fitting HRTF was set up using Unity with the SOFAlizer plugin [5] which facilitated switching between different HRTF profiles with ease.  The HRTF contenders were obtained from the open-source databases by MIT, CIPIC and RIEC [6, 7, 8].  A total of 30 HRTF profiles were used during this evaluation.  A completely accurate best fitting HRTF profile match was not found.  In essence, of the HRTF profiles examined, none overcame the front/back discrepancy described earlier.  Furthermore, this process was rather tedious as each HRTF had to be tested with the sound source located at varying position – the effort required did not equate to an extremely well fitted HRTF.
+The second approach of using the application that generates an individualised HRTF based off of measurements of the ear was a much more concise process.  After providing the input of a few measurements of the ear and head the application produced a comparable HRTF to that of the first individualisation method; although it was still not perfect.  Nevertheless, as the focus for this project was on the feedback algorithm, and not the accuracy of individualisation of an HRTF profile, the HRTF obtained in the second method was used in the subsequent feedback algorithm testing.
+Spatialised sounds were able to be accurately created and played through the bone conduction headphones.  The position of the landmarks (ArUco markers) located along the side of the user could be successfully located solely based-off of the sound.  Visual feedback provides assistance in locating sound sources from the front and back due to the front/back discrepancy, however, the use of a well-fitted HRTF profile would mitigate this issue.
+The distances, in feet, between the user and the landmark(s) were also announced to the users.  This was computed by simply calculating the norm of the vector between the user body and the landmark.  It was found that if more than three identified obstacles (often will be the case), including their distances, were announced created a larger load on the senses and could potentially disengage the users.  Therefore, in order to reduce the cognitive load, the algorithm was updated such that only obstacles within 10 feet of the body were announced.  It was also found that the listing of landmark names along with their distances is slow and users can move rather quickly.  Thus, the distances of the obstacles from the user were only announced every third iteration of the loop and not every single time the obstacles were listed in order to ensure adequate and accurate auditory feedback is provided.
+**Visual Feedback System**
+The horizontal and vertical FOVs had to be fine-tuned to accurately reflect the landmark positions in real life.  The final FOVs are:
+Horizontal FOV = 80°
+Vertical FOV = 60°.
+The figure below shows that the landmarks within the user’s FOV were successfully placed and shown on the virtual reality headset display.
+{{ :courses:456:2022:projects:456-2022-10:vr.png?400 |}}
-As can be seen the IRB and participant availability dependency were not achieved.  This was due to the fact that the existing IRB protocol only includes recruiting participants from APL or Argus II users.  The developed supplementary feedback systems, though integrated, are not in a complete enough state to be tested with Argus II users at this stage.  In addition, there was not enough time during this project timeline to execute the fallback plan for participant testing which was to test the developed integrated system with APL employees.  As seen above under the deliverables, the maximum deliverable was adjusted due to these time constraints and IRB dependency not being met.
 ======Milestones and Status ======
@@ Line 189: / Line 225: @@
   * Project Checkpoint
     * {{ :courses:456:2022:projects:456-2022-10:checkpoint_presentation.pdf | Project Checkpoint Presentation}}
-  * Project Final Presentation
+  * Project Final Poster
-    * {{:courses:456:2022:projects:456-2022-01:final_poster_pdf.pdf|PDF of Poster}}
+    * {{ :courses:456:2022:projects:456-2022-10:poster.pdf | Final Poster}}
   * Project Final Report
     * {{ :courses:456:2022:projects:456-2022-10:finalreport.pdf | Final Report}}

courses/456/2022/projects/456-2022-10/project-10.1651346496.txt.gz · Last modified: 2022/04/30 19:21 by achen134