Kevin Yu

Contact

Room: Forschungsgruppe MITI, second floor (1. Geschoss)

            Trogerstraße 26, 81675 Munich

Phone: +49 (0)89 4140-7388

Email: kevin.yu(at)tum.de

You can contact me in english and german

Research Interests

  • Augmented Reality and Virtual Reality for Medical Applications
  • Visualization of Medical Data
  • User Interaction in XR and 3DUI

Current Projects

ARTEKMED

Don't miss out the project homepage: http://www.artekmed.de/

The goal of ARTEKMED is to provide paramedics and young doctors the possibility to allow a remote expert to help them in unfamiliar situations. By using Augmented Reality and head-mounted displays (i.e. Microsoft HoloLens), they can see 3D annotations and live guidance from the remote expert. The remote expert sees the 3D reconstruction of the local environment including patient data and real-time sensor data, in order to provide conscientious teleconsultation and annotations. The 3D reconstruction is made possible by using a multi-camera setup at the emergency site, either using the built-in cameras of the head-mounted displays itself or cameras inside the ambulance, to reconstruct the environment of the patient and send this data to the remote expert.

ArtekMed is currently in mid-development stage and is operated in tight collaboration with the NarVis group (http://campar.in.tum.de/Main/NarvisLabNew)  and the Institute for Emergency Medicine (https://www.inm-online.de/de/)

Open topics:

Development of a "Surgical Cockpit", including UI and annotations:

  • Novel concepts of teleconsultation
  • Exploring different annotation methodologies for teleconsultation

Please contact me, if you are interested in this project. Preferably, you have experience with Unity / C# or AR/VR.
If you have cool ideas for AR/VR in a clinical environment, you are welcome to discuss it with us. We can provide HoloLens, 3D Printers, clinical partners, and more. 

To see this content (source: domain.net), you have to click accept. We would like to point out that by accepting this iframe data could be transmitted to third parties or cookies could be saved. For more information, see our privacy policy.
To see this content (source: domain.net), you have to click accept. We would like to point out that by accepting this iframe data could be transmitted to third parties or cookies could be saved. For more information, see our privacy policy.

Augmented Mirror

[Link to Paper]

A recurrent problem in egocentric Augmented Reality (AR) applications is the misestimation of depth. Providing alternative views from non-egocentric perspectives can convey useful information for applications that require the correct judgment of depth as it is in the case of placement and alignment of virtual and real content, but also for exploration and visualization tasks.In this paper, we introduce Augmented Mirrors. Through the integration of a real mirror, our approach is capable to reflect changes of the real and virtual content of an AR application while users benefit from the perceptual advantages of using mirrors. Our concept, simple yet effective, only requires tracking the user and mirror poses with the accuracy demanded by a specific application. To showcase the potential and flexibility of the Augmented Mirrors, we present and discuss multiple examples ranging from alignment, exploration, spatial understanding, and selective content visualization using different AR-enabled devices and tracking technologies. We envision the Augmented Mirrors as a new and valuable concept that can be used in applications that benefit from additional viewpoints and require the simultaneous visualization of real and virtual content.

This project is with collaboration with the Chair of Computer Aided Medical Procedures by Prof. Nassir Navab.

Open topics:
This can be topic of a bachelor or master thesis. Knowledge in Computer Vision and coding are required.

To see this content (source: domain.net), you have to click accept. We would like to point out that by accepting this iframe data could be transmitted to third parties or cookies could be saved. For more information, see our privacy policy.

Visualization of Sterility

Open Topics:
This can be topic of a bachelor or master thesis. Knowledge in coding (C#) and Unity is recommended.

To see this content (source: domain.net), you have to click accept. We would like to point out that by accepting this iframe data could be transmitted to third parties or cookies could be saved. For more information, see our privacy policy.

Students

Student Type Topic
Thomas Wegele Bachelor's Thesis

Development of a Concept for an Augmented Reality Telecommunication Application for Medical Purposes

Alexander Burghardt Bachelor's Thesis Efficient Real-time Point Cloud Transfer and Virtual Reality Rendering System
Thomas Nibler IDP (ongoing) tbd
Vanessa Kern Bachelor's Thesis (ongoing) tbd
To see this content (source: domain.net), you have to click accept. We would like to point out that by accepting this iframe data could be transmitted to third parties or cookies could be saved. For more information, see our privacy policy.

News

To see this content (source: domain.net), you have to click accept. We would like to point out that by accepting this iframe data could be transmitted to third parties or cookies could be saved. For more information, see our privacy policy.

Honorable Mention Demo Award for work on Magnoramas!

IEEE VR 2021 was super exciting and well-done in nearly all aspects. I am happy to announce that our demo on the work of Magnoramas has received the Honorable Mention Demo Award. I want to thanks everyone who was involved in this project and hope to achieve much more in the future.

To see this content (source: domain.net), you have to click accept. We would like to point out that by accepting this iframe data could be transmitted to third parties or cookies could be saved. For more information, see our privacy policy.

Don't miss our talks on IEEE VR 2021!

Exciting news! Two papers from the ArtekMed project are accepted to this years IEEE VR 2021 conference.
Please do not miss this chance and visit the online presentation of our work.
- Conference Paper: 
Magnoramas: Magnifying Dioramas for Precise Annotations in Asymmetric 3D Teleconsultation
- 3DUI Contest: 
Real-time Mixed Reality Teleconsultation for Intensive Care Units in Pandemic Situations

If you are not convinced yet, please take a look at the teaser video.

 

Aug 5th - Aug 16th: Medical Augmented Reality Summer School 2019 in Balgrist, Zurich

The focus of the Medical Augmented Reality Summer School Zurich is to teach the new generation of scientists the basics, challenges and recent advantages of Augmented Reality as well as to strengthen the connection between medicine, science and industry. In the first week, leading scientists of the field will present keynotes and insights from a medical and technical point of view. A condensed version of the TU Munich and JHU Baltimore lecture “Medical Augmented Reality” will provide profound theoretical knowledge of the topic. In the second week, the participants will develop their own Medical AR solutions using cutting-edge technologies in interdisciplinary teams.

To see this content (source: domain.net), you have to click accept. We would like to point out that by accepting this iframe data could be transmitted to third parties or cookies could be saved. For more information, see our privacy policy.

At this amazing event, my group developed ARISE during the one-week hackathon. ARISE is an AR parkour for the physical rehabilitation of patients after a stroke incident. It combines the SLAM capability of the HoloLens 2 with the mobile motion tracking system Xsens. The result is a highly powerful system for generating large and adaptiv AR parkours with integrated patient feedback.

Publications

2021
  • [Utility Model] Yu K, Gomez A, Navab N, Winkler A (all authors equally own this patent). Vorrichtung eines Spiegels mit Erweiterter Realität
  • [Poster] Roth D, Yu K, Pankratz F, Gorbachev G, Keller A, Lazarovici M, Wilhelm D, Weidert S, Navab N, Eck U. Real-time Mixed Reality Teleconsultation for Intensive Care Units in Pademic Situations, 3DUI ContestIEEE VR 2021, 2021 Mar,
  • Yu K, Winkler A, Pankratz F, Lazarovici M, Wilhelm D, Eck U, Roth D, Navab N. Magnoramas: Magnifying Dioramas for Precise Annotations in Asymmetric 3D Teleconsultation IEEE VR 2021, 2021 Mar, Honorable Mention Demo Award
2020
  • Yu K*, Gomez A*, Winkler A*, Roth D, Eck U, Navab N. Augmented Mirrors IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2020 Nov (*Shared Authorship)
  • Yu K, Wegele T, Ostler D, Wilhelm D, Feußner H. EyeRobot: Enabling Telemedicine Using a Robot Arm and a Head-Mounted Display. Current Directions in Biomedical Engineering. 2020 Sep 17;6(1).
  • Yu K*, Unberath M*, Barmaki R, Johnson A, Navab N Augment Yourself: Mixed Reality Self-Augmentation Using Optical See-through Head-mounted Displays and Physical Mirrors arXiv preprint arXiv:2007.02884 (*Shared Authorship)
  • Yu K*, Held JPO*, Pyles C, Veerbeek JM, Bork F, Heining SM, Navab N, Luft AR Augmented Reality–Based Rehabilitation of Gait Impairments: Case Report. JMIR Mhealth Uhealth 2020;8(5):e17804, URL: https://mhealth.jmir.org/2020/5/e17804, DOI: 10.2196/17804, PMID: 32452815 (*Shared Authorship)
  • Hartwig R, Ostler D, Feußner H, Berlet M, Yu K, Rosenthal J, Wilhelm D. COMPASS: Localization in Laparoscopic Visceral Surgery Current Directions in Biomedical Engineering 6.1 (2020)
2019
  • Guo, Z., Yu, K., Pearlman, R., Navab, N., & Barmaki, R. (2019). Collaboration Analysis Using Deep Learning. arXiv preprint arXiv:1904.08066.
  • Barmaki, R., Yu, K., Pearlman, R., Shingles, R., Bork, F., Osgood, G. M., & Navab, N. (2019). Enhancement of Anatomical Education Using Augmented Reality: An Empirical Study of Body Painting. Anatomical sciences education.
2018
  • Deib, G., Johnson, A., Unberath, M., Yu, K., Andress, S., Qian, L., ... & Gailloud, P. (2018). Image Guided Percutaneous Spine Procedures Using an Optical See-Through Head Mounted Display: Proof of Concept and Rationale. Journal of neurointerventional surgery, 10(12), 1187-1191.
  • Andress, S., Johnson, A., Unberath, M., Winkler, A., Yu, K., Fotouhi, J., ... & Navab, N. (2018, March). On-the-fly Augmented Reality for Orthopaedic Surgery Using a Multi-Modal Fiducial. In Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling (Vol. 10576, p. 105760H). International Society for Optics and Photonics.
  • Yu, K., Barmaki, R., Unberath, M., Mears, A., Brey, J., Chung, T. H., & Navab, N. (2018, May). On the Accuracy of Low-Cost Motion Capture Systems for Range of Motion Measurements. In Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications (Vol. 10579, p. 105790G). International Society for Optics and Photonics.
2017
  • Qian, L., Unberath, M., Yu, K., Fuerst, B., Johnson, A., Navab, N., & Osgood, G. (2017). Towards Virtual Monitors for Image Guided Interventions-Real-Time Streaming to Optical See-Through Head-Mounted Displays. arXiv preprint arXiv:1710.00808.
  • Bork, F., Barmaki, R., Eck, U., Yu, K., Sandor, C., & Navab, N. (2017, October). Empirical Study of Non-Reversing Magic Mirrors for Augmented Reality Anatomy Learning. In 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 169-176). IEEE.

Google Scholar: https://scholar.google.de/citations?user=DXhMcasAAAAJ
ResearchGate: https://www.researchgate.net/profile/Kevin_Yu22