Kevin Yu

Contact

Room: Forschungsgruppe MITI, second floor (1. Geschoss)

            Trogerstraße 26, 81675 Munich

Phone: +49 (0)89 4140-7388

Email: kevin.yu(at)tum.de

You can contact me in english and german

Research Interests

  • Augmented Reality and Virtual Reality for Medical Applications
  • Visualization of Medical Data
  • User Interaction in XR and 3DUI

Current Projects

ARTEKMED

Don't miss out the homepage of the project: http://www.artekmed.de/

The goal of ARTEKMED is to provide paramedics and young doctors the possibility to allow a remote expert to help them in unfamiliar situations. By using Augmented Reality and head-mounted displays (i.e. Microsoft HoloLens), they can see 3D annotations and live guidance from the remote expert. The remote expert sees the 3D reconstruction of the local environment including patient data and real-time sensor data, in order to provide conscientious teleconsultation and annotations. The 3D reconstruction is made possible by using a multi-camera setup at the emergency site, either using the built-in cameras of the head-mounted displays itself or cameras inside the ambulance, to reconstruct the environment of the patient and send this data to the remote expert.

ArtekMed is currently in mid-development stage and is operated in tight collaboration with the NarVis group (http://campar.in.tum.de/Main/NarvisLabNew)  and the Institute for Emergency Medicine (https://www.inm-online.de/de/)

Open topics:

Development of a "Surgical Cockpit", including UI and annotations:

  • Novel concepts of teleconsultation (preferably asymmetric)
  • User-Study - VR User Interface for a remote expert in a teleconsultation setup
    • How to efficiently and intuitively display vital and patient information? (e.g. 2D vs 3D visualization, location and opacity of sensor data, workflow-based information display) 
    • What visualization is best? (e.g. number values vs abstractized visualization)
    • How can we efficiently integrate Eye-Tracking into the UI of the Surgical Cockpit?
  • Exploring different annotation methodologies for teleconsultation
    • Novel methods for creating annotations, especially suited for medical staff
    • Creation of various annotations in VR and chance to prototype own annotation methods (e.g. 3D lines, 3D voice messages, pointers, ...)
    • Do the same methods in VR work for AR?

 

Please contact me, if you are interested in this project. Preferably, you have experience with Unity / C# or AR/VR.
If you have cool ideas for AR/VR in a clinical environment, you are welcome to discuss it with us. We can provide HoloLens, 3D Printers, clinical partners, and more. 


Past Students

Student Type Topic
Thomas Wegele Bachelor's Thesis Development of a Concept for an Augmented Reality Telecommunication Application for Medical Purposes

Credits to Thomas Wegele.


News

Honorable Mention Demo Award for work on Magnoramas!

IEEE VR 2021 was super exciting and well-done in nearly all aspects. I am happy to announce that our demo on the work of Magnoramas has received the Honorable Mention Demo Award. I want to thanks everyone who was involved in this project and hope to achieve much more in the future.

Don't miss our talks on IEEE VR 2021!

Exciting news! Two papers from the ArtekMed project are accepted to this years IEEE VR 2021 conference.
Please do not miss this chance and visit the online presentation of our work.
- Conference Paper: 
Magnoramas: Magnifying Dioramas for Precise Annotations in Asymmetric 3D Teleconsultation
- 3DUI Contest: 
Real-time Mixed Reality Teleconsultation for Intensive Care Units in Pandemic Situations

If you are not convinced yet, please take a look at the teaser video.

 

Aug 5th - Aug 16th: Medical Augmented Reality Summer School 2019 in Balgrist, Zurich

The focus of the Medical Augmented Reality Summer School Zurich is to teach the new generation of scientists the basics, challenges and recent advantages of Augmented Reality as well as to strengthen the connection between medicine, science and industry. In the first week, leading scientists of the field will present keynotes and insights from a medical and technical point of view. A condensed version of the TU Munich and JHU Baltimore lecture “Medical Augmented Reality” will provide profound theoretical knowledge of the topic. In the second week, the participants will develop their own Medical AR solutions using cutting-edge technologies in interdisciplinary teams.

At this amazing event, my group developed ARISE during the one-week hackathon. ARISE is an AR parkour for the physical rehabilitation of patients after a stroke incident. It combines the SLAM capability of the HoloLens 2 with the mobile motion tracking system Xsens. The result is a highly powerful system for generating large and adaptiv AR parkours with integrated patient feedback.

Publications

2021
  • Yu K, Winkler A, Pankratz F, Lazarovici M, Wilhelm D, Eck U, Roth D, Navab N. Magnoramas: Magnifying Dioramas for Precise Annotations in Asymmetric 3D Teleconsultation IEEE VR 2021, 2021 Mar, Honorable Mention Demo Award
2020
  • Yu K*, Gomez A*, Winkler A*, Roth D, Eck U, Navab N. Augmented Mirrors IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2020 Nov (*Shared Authorship)
  • Yu K, Wegele T, Ostler D, Wilhelm D, Feußner H. EyeRobot: Enabling Telemedicine Using a Robot Arm and a Head-Mounted Display. Current Directions in Biomedical Engineering. 2020 Sep 17;6(1).
  • Yu K*, Unberath M*, Barmaki R, Johnson A, Navab N Augment Yourself: Mixed Reality Self-Augmentation Using Optical See-through Head-mounted Displays and Physical Mirrors arXiv preprint arXiv:2007.02884 (*Shared Authorship)
  • Yu K*, Held JPO*, Pyles C, Veerbeek JM, Bork F, Heining SM, Navab N, Luft AR Augmented Reality–Based Rehabilitation of Gait Impairments: Case Report. JMIR Mhealth Uhealth 2020;8(5):e17804, URL: https://mhealth.jmir.org/2020/5/e17804, DOI: 10.2196/17804, PMID: 32452815 (*Shared Authorship)
2019
  • Guo, Z., Yu, K., Pearlman, R., Navab, N., & Barmaki, R. (2019). Collaboration Analysis Using Deep Learning. arXiv preprint arXiv:1904.08066.
  • Barmaki, R., Yu, K., Pearlman, R., Shingles, R., Bork, F., Osgood, G. M., & Navab, N. (2019). Enhancement of Anatomical Education Using Augmented Reality: An Empirical Study of Body Painting. Anatomical sciences education.
2018
  • Deib, G., Johnson, A., Unberath, M., Yu, K., Andress, S., Qian, L., ... & Gailloud, P. (2018). Image Guided Percutaneous Spine Procedures Using an Optical See-Through Head Mounted Display: Proof of Concept and Rationale. Journal of neurointerventional surgery, 10(12), 1187-1191.
  • Andress, S., Johnson, A., Unberath, M., Winkler, A., Yu, K., Fotouhi, J., ... & Navab, N. (2018, March). On-the-fly Augmented Reality for Orthopaedic Surgery Using a Multi-Modal Fiducial. In Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling (Vol. 10576, p. 105760H). International Society for Optics and Photonics.
  • Yu, K., Barmaki, R., Unberath, M., Mears, A., Brey, J., Chung, T. H., & Navab, N. (2018, May). On the Accuracy of Low-Cost Motion Capture Systems for Range of Motion Measurements. In Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications (Vol. 10579, p. 105790G). International Society for Optics and Photonics.
2017
  • Qian, L., Unberath, M., Yu, K., Fuerst, B., Johnson, A., Navab, N., & Osgood, G. (2017). Towards Virtual Monitors for Image Guided Interventions-Real-Time Streaming to Optical See-Through Head-Mounted Displays. arXiv preprint arXiv:1710.00808.
  • Bork, F., Barmaki, R., Eck, U., Yu, K., Sandor, C., & Navab, N. (2017, October). Empirical Study of Non-Reversing Magic Mirrors for Augmented Reality Anatomy Learning. In 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 169-176). IEEE.

Google Scholar: https://scholar.google.de/citations?user=DXhMcasAAAAJ
ResearchGate: https://www.researchgate.net/profile/Kevin_Yu22