Alignment of the holographic model and the real object in neuronavigation using augmented reality (AR): accuracy analysis of three methods

Introduction. Augmented reality (AR) technologies allow a surgeon to overlay three-dimensional virtual models onto real objects, enhancing visualization and facilitating the performance of complex surgeries. The primary objective of using AR in surgery is the precise alignment of the virtual model with the patient’s real anatomy, enabling the surgeon to see key anatomical structures as a projection onto or inside the body.

Materials and methods. The study utilized Microsoft Hololens 2 HMD (head-mounted display) glasses and the “Medgital” software for AR navigation. This article presents the results of a technical comparison of three methods for aligning AR 3D models with a patient’s head: alignment using craniometric points, the use of a QR code and a 3D printed frame and the application of pre-programmed points with a 3D pointer. The accuracy of fiducial point registration was assessed using the fiducial registration error (FRE) metric, and the execution time was measured for each method, allowing for the determination of their advantages and limitations.

Results. The mean FRE for alignment using craniometric points was 1.1 ± 0.3 cm, for the method using a QR code and a 3D printed frame it was 0.4 ± 0.3 cm, and for the method using pre-programmed points and a 3D pointer it was 0.7 ± 0.6 cm. The average alignment time was 180, 5, and 100 seconds, respectively. It was found that the method using the QR code and the 3D frame offered the highest accuracy, making it preferable for most clinical situations.

Conclusion. Each method has its own strengths and weaknesses; the choice depends on the clinical situation, available equipment, and required accuracy. The integration of AR technologies with surgical techniques improves the safety and precision of operative interventions.

1. Kim Y, Kim H, Kim YO. Virtual reality and augmented reality in plastic surgery: A review. Arch Plast Surg. 2017; 44(3): 179-187. https://doi.org/10.5999/aps.2017.44.3.179

2. Azuma RT. A survey of augmented reality. Vol. 6. Presence: Teleoperators and virtual environments. 1997. doi.org: 10.1162/pres.1997.6.4.355

3. Mahmoud N, Grasa ÓG, Nicolau SA, Doignon C, Soler L, Marescaux J, et al. On-patient see-through augmented reality based on visual SLAM. Int J Comput Assist Radiol Surg. 2017; 12(1): 1-11. https://doi.org/10.1007/s11548-016-1444-x

4. Mitsuno D, Ueda K, Hirota Y, Ogino M. Effective application of mixed reality device HoloLens: Simple manual alignment of surgical field and holograms. Plast Reconstr Surg. 2019; 143(2): 647-651. https://doi.org/10.1097/PRS.0000000000005215. Erratum in: Plast Reconstr Surg. 2019; 143(4): 1283. https://doi.org/10.1097/PRS.0000000000005770

5. Lia H, Paulin G, Yi N, Haq H, Emmanuel S, Ludig K, et al. HoloLens in suturing training. Image-Guided Procedures, Robotic Interventions, and Modeling. 2018: 69. https://doi.org/10.1117/12.2293934

6. Kilgus T, Heim E, Haase S, Prüfer S, Müller M, Seitel A, Fangerau M, et al. Mobile markerless augmented reality and its application in forensic medicine. Int J Comput Assist Radiol Surg. 2015; 10(5): 573-586. https://doi.org/10.1007/s11548-014-1106-9

7. Wang H, Wang F, Leong AP, Xu L, Chen X, Wang Q. Precision insertion of percutaneous sacroiliac screws using a novel augmented reality-based navigation system: A pilot study. Int Orthop. 2016; 40(9): 1941-1947. https://doi.org/10.1007/s00264-015-3028-8

8. Van Krevelen DWF, Poelman R. A survey of augmented reality technologies, applications and limitations. Int J Virtual Real. 2010; 9(2): 1–20. https://doi.org/10.20870/IJVR.2010.9.2.2767

9. Badiali G, Ferrari V, Cutolo F, Freschi C, Caramella D, Bianchi A, et al. Augmented reality as an aid in maxillofacial surgery: Validation of a wearable system allowing maxillary repositioning. J Craniomaxillofac Surg. 2014; 42(8): 1970-1976. https://doi.org/10.1016/j.jcms.2014.09.001

10. Tepper OM, Rudy HL, Lefkowitz A, Weimer KA, Marks SM, Stern CS, et al. Mixed reality with HoloLens: Where virtual reality meets augmented reality in the operating room. Plast Reconstr Surg. 2017; 140(5): 1066–1070. https://doi.org/10.1097/PRS.0000000000003802

11. Yoon JW, Chen RE, Kim EJ, Akinduro OO, Kerezoudis P, Han PK, et al. Augmented reality for the surgeon: Systematic review. Int J Med Robot. 2018; 14(4): e1914. https://doi.org/10.1002/rcs.1914

12. Kuhlemann I, Kleemann M, Jauer P, Schweikard A, Ernst F. Towards X-ray free endovascular interventions – Using HoloLens for on-line holographic visualisation. Healthcare Technology Letters. Institution of Engineering and Technology. 2017: 184–187.