GMSJ Logo

Innovations

Vol. 2 No. 1 (2026)

Utilizing a Voice-Guided Mixed Reality and Artificial Intelligence based platform for Difficult Intubation Simulation in Austere Teaching Environments

Submitted
26 November 2025
Published
09-04-2026

Abstract

Background: Intubating a blood-contaminated airway is a high acuity, low occurrence procedure.  In resource-limited environments, a high-fidelity bloody airway simulation is restricted by cost, equipment, and a lack of expert feedback. To bridge this gap, we developed a high-fidelity artificial intelligence (AI) and augmented reality (AR) informed simulation tool. Inside an AR headset, an overlay is generated over a standard airway manikin. The user then proceeds through a bloody airway scenario with integrated instructional feedback videos.

Methods: We utilized a Unity AR application integrated into an AR Headset that interacts with a laryngoscope and removable pressure, light, and proximity sensors inside a manikin. The sensors and laryngoscope are connected to a Raspberry Pi that transmits sensor and video data via WebSocket to the headset. The headset application also processes audio via Meta’s Wit.AI service for transcription and intent recognition. The application handles simulation logic, sensor / video data handling, and the AR guidance display. A decision tree algorithm enables user interaction and feedback.

Results: Both non-medical users at the 2024 DALI exhibition and individuals with emergency medicine backgrounds at the 2025 Society for Academic Emergency Medicine Innovation exhibit reported that the device was educational for difficult airway management. However, challenges have arisen in terms of stable connectivity with the Raspberry Pi under certain internet network conditions. 

Conclusions: Utilizing readily available AI and AR technologies, a simulation device can be developed to teach difficult airway management in resource-limited areas, providing both experience and constructive feedback from experts.

References

  1. Wang, H. E., Balasubramani, G. K., Cook, L. J., Lave, J. R., & Yealy, D. M. (2010). Out-of-hospital endotracheal intubation experience and patient outcomes. Annals of Emergency Medicine, 55(6), 527–537. https://doi.org/10.1016/j.annemergmed.2009.12.016
  2. Patel, S. H., Rudolf, F., Schwartz, K., Gabriel, R. A., Hastings, R. H., Daniel, M., & Suresh, P. J. (2024). Enhancing laryngoscopy mastery: The impact of autonomous practice with feedback-providing simulators. A&A Practice, 18(7), e01825. https://doi.org/10.1213/XAA.0000000000001825
  3. Fei, G. T., Guo, A. W., Booth, T., Kiss, E., Hallac, R. R., Anderson, A., & Wang, Z. J. (2024). Development of an augmented reality system for tracheal intubation guidance of airway management. Proceedings of SPIE, The International Society for Optical Engineering, 12818, 1281808. https://doi.org/10.1117/12.3005990
  4. Steffensen, T. L., Bartnes, B., Fuglstad, M. L., Auflem, M., & Steinert, M. (2023). Playing the pipes: Acoustic sensing and machine learning for performance feedback during endotracheal intubation simulation. Frontiers in Robotics and AI, 10, 1218174. https://doi.org/10.3389/frobt.2023.1218174
  5. Miller, M. E., Witte, D., Lina, I., Walsh, J., Rameau, A., & Bhatti, N. I. (2025). Development of machine learning copilot to assist novices in learning flexible laryngoscopy. The Laryngoscope, 135(3), 1046–1053. https://doi.org/10.1002/lary.31812
  6. Dias, P. L., Greenberg, R. G., Goldberg, R. N., Fisher, K., & Tanaka, D. T. (2021). Augmented reality-assisted video laryngoscopy and simulated neonatal intubations: A pilot study. Pediatrics, 147(3), e2020027228. https://doi.org/10.1542/peds.2020-027228
  7. Zhao, S., Xiao, X., Wang, Q., Zhang, X., Li, W., Soghier, L., & Hahn, J. (2020, November). An intelligent augmented reality training framework for neonatal endotracheal intubation. 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 672–681. https://doi.org/10.1109/ISMAR50242.2020.00092
  8. Jain, S., Barua Chowdhury, B. D., Mosier, J. M., Subbian, V., Hughes, K., & Son, Y. J. (2025). Design and development of an integrated virtual reality (VR)-based training system for difficult airway management. IEEE Journal of Translational Engineering in Health and Medicine, 13, 49–60. https://doi.org/10.1109/JTEHM.2024.3490989
  9. Duffy, C. C., Bass, G. A., Yi, W., Rouhi, A., Kaplan, L. J., & O'Sullivan, E. (2024). Teaching airway management using virtual reality: A scoping review. Anesthesia & Analgesia, 138(4), 782–793. https://doi.org/10.1213/ANE.0000000000006733
  10. Green, A., & Hug, M. (2023). Simulation training and skill assessment in EMS. In StatPearls. StatPearls Publishing. https://www.ncbi.nlm.nih.gov/books/NBK570595/
  11. Davis, L., Ha, Y., Frolich, S., Martin, G., Meyer, C., Pettitt, B., Norfleet, J., Lin, K. C., & Rolland, J. P. (2002). Augmented reality and training for airway management procedures. Studies in Health Technology and Informatics, 85, 121–126.
  12. Petersen, C., Rush, S. C., Gallo, I., Dalere, B., Staak, B., Moore, L., Kerr, W., Chandler, M., & Smith, W. (2017). Optimization of simulation and moulage in military-related medical training. Journal of Special Operations Medicine, 17(3), 74–80.
  13. Stone, R. J., Guest, R., Mahoney, P., Mluduch, M., & Carr, K. (2017). A ‘mixed reality’ simulator concept for future medical emergency response team training. BMJ Military Health, 163(4), 280–287. https://doi.org/10.1136/jramc-2017-000783
  14. Rabotin, A., Glick, Y., Gelman, R., Ketko, I., Taran, B., Fink, N., & Furer, A. (2023). Practicing emergency medicine in the metaverse: A novel mixed reality casualty care training platform. Surgical Innovation, 30(5), 586–594. https://doi.org/10.1177/15533506231174092