- Open Access
Augmented reality-based navigation system applied to tibial bone resection in total knee arthroplasty
Journal of Experimental Orthopaedics volume 6, Article number: 44 (2019)
This pilot study was performed to examine the accuracy of the AR-KNEE system, an imageless navigation system using augmented reality (AR) technology for total knee arthroplasty. The AR-KNEE system enables the surgeon to view information from the navigation superimposed on the surgical field on a smartphone screen in real time.
Using the AR-KNEE system, one surgeon resected 10 tibial sawbones with viewing the tibial axis and aiming varus/valgus, posterior slope, internal/external rotation angles, and resection level superimposed on the surgical field. We performed computed tomography of the resected sawbones and measured the varus/valgus, posterior slope, and internal/external rotation angles using a designated computer software. The thickness of the resected bone was measured using digital calipers.
The absolute differences between the values displayed on the smartphone screen and the measurement values for varus/valgus, posterior slope, internal/external rotation angles, and thickness of the resected bone were 0.5° ± 0.2°, 0.8° ± 0.9°, 1.8° ± 1.5°, and 0.6 mm ± 0.7 mm, respectively.
This pilot study using sawbones suggested that the AR-KNEE system may provide reliable accuracy for coronal, sagittal, and rotational alignment in tibial bone resection during total knee arthroplasty.
Although computer navigation for total knee arthroplasty (TKA) can improve limb alignment and may reduce the revision rate (de Steiger et al. 2015), its lack of cost-effectiveness remains a challenge to its widespread adoption (Novak et al. 2007; Slover et al. 2008). The need for the surgeon to take their eyes off the surgical field to view the monitor is another issue that must be addressed in the further development of computer navigation.
In total hip arthroplasty (THA), Ogawa et al. developed a novel navigation system using augmented reality (AR) technology, in which the surgeon uses a free application installed on a smartphone (Ogawa et al. 2018). AR-based navigation allows the surgeon to view the information from the navigation superimposed on the surgical field through the smartphone. Putting the smartphone in a sterilized case enables the surgeon to use AR-based navigation without taking their eyes off the surgical field in a sterile setting. There have been no previous reports regarding the use of an AR-based navigation system for TKA.
For TKA, we developed an AR-based navigation system (AR-KNEE system) that was applied to tibial bone resection. As the surgeon uses an application installed on their own smartphone, no expensive equipment is required when using the AR-KNEE system. The surgeon can easily determine the target angles of varus/valgus and posterior slope by pushing the icons on the smartphone display. The AR technology allows the surgeon to view these angles as reference lines superimposed on the real surgical field on the smartphone display.
In this pilot study, we introduce the AR-KNEE system for tibial bone resection. The aim of this preclinical study was to assess the accuracy of the AR-KNEE system in tibial bone resection using sawbones. We hypothesized that the AR-KNEE system would provide reliable accuracy for coronal, sagittal, and rotational alignment in tibial bone resection during TKA.
AR technology projects computer-generated images onto the real world (Ogawa et al. 2018). The AR-KNEE system allows the surgeon to view the tibial axis superimposed on the surgical field through the display of the smartphone. The AR-KNEE system also enables the surgeon to view the aiming varus/valgus angle and posterior slope angle superimposed on the surgical field (Fig. 1). Similar to conventional computer navigation, the AR-KNEE system provides real-time information during surgery and intraoperative feedback.
The AR-KNEE system consists of a guide marker, pointer, oval marker, and the surgeon’s own smartphone (Fig. 2). The markers are made of acrylonitrile-butadiene-styrene resin and created using a three-dimensional printer for home use.
The guide marker is fixed to the proximal tibia through the same skin incision as the surgical approach (Fig. 1). The guide marker is designed to be placed proximal to the superior margin of pes anserinus. According to the surgeon’s preference, the guide marker can be fixed to the proximal tibia through separate small incision.
The smartphone camera recognizes the guide marker (Fig. 1). The surgeon registers bone landmarks using the pointer (Fig. 3). The smartphone mathematically interprets the positional relationships between the guide marker and bone landmarks, and creates a tibial coordinate system. The position of the smartphone in relation to the lower limb during registration is not associated with the accuracy of the acquired data. The smartphone displays the reference lines including tibial long and anteroposterior axes of the tibial coordinate system (Figs. 1, 3).
After bone resection, the surgeon can confirm the varus/valgus, posterior slope, and rotation angles using the oval marker (Fig. 4).
Surgical technique of AR-KNEE system
AR-KNEE utilizes tibial long and anteroposterior axes to uniquely determine the positions of the points in the surgical field. Three lines constituting the 3-dimensional coordinate system of AR-KNEE are (1) the tibial long axis, (2) the tibial anteroposterior axis, and (3) the cross-product of these two tibial axes. These tibial axes are created by registration of bone landmarks. Using the pointer, the following bone landmarks are registered: (1) the most prominent point of the medial malleolus, (2) the most prominent point of the lateral malleolus, (3) the tibial center on the tibial plateau, (4) the mediolateral center of the tibial insertion of the posterior cruciate ligament, (5) the medial border of the patellar tendon attachment, (6) the medial compartment of the tibial plateau, and (7) the lateral compartment of the tibial plateau.
The tibial long axis is determined from both the ankle center and the tibial center on the tibial plateau. In the AR-KNEE system, the center of the ankle is determined from the medial and lateral malleolus. The definition of the ratio of the ankle center from the lateral malleolus to the medial malleolus can be changed to suit the surgeon’s preference. In this study, the ankle center was calculated by dividing the digitized transmalleolar axis according to the ratio of 55% lateral to 45% medial (Kim et al. 2005). The definition of the tibial center of the tibial plateau can be determined according to the surgeon’s preference. In this study, the tibial center on the tibial plateau was defined as the bisection of the transverse tibial axis in this study (Stiehl 2007).
The anteroposterior axis of the tibia was determined by a line connecting the middle of the posterior cruciate ligament to the medial border of the patellar tendon attachment (Akagi et al. 2004).
The registration of medial and lateral compartments of the tibial plateau is used for the resection level of the tibia allowing the surgeon to view the proximal tibia resection level. A single point on each plateau is used to determine the resection level. The AR-KNEE system calculates the length of the line perpendicular from the reference point to the resection plane.
After completing registration, the AR-KNEE system enables the surgeon to view the reference lines superimposed on the tibia on their smartphone display (Fig. 1). The surgeon can also view the angles of varus/valgus and posterior slope on the display. The surgeon fixes the tibial resection block while viewing these angles. The tibia is resected through the slit of the tibial resection block using a bone saw in the standard manner. Placing the oval marker on the cut bone surface, the surgeon can intraoperatively confirm the angles of varus/valgus, posterior slope, and rotation (Fig. 4). The oval marker has a slit in the central part, which the surgeon use to mark the reference line of tibial rotational alignment using the slit (Fig. 2).
Accuracy verification of AR-KNEE system using sawbones
This study was approved by the institutional ethics board. A total of 10 pairs of tibia and fibula sawbones were used to assess the accuracy of the AR-KNEE system (Fig. 5). A single knee surgeon (ST) resected the proximal tibia using the AR-KNEE system. The target values of tibial resection in this study were as follows: (1) perpendicular to the tibial long axis in the coronal plane; (2) 5° posterior slope in the sagittal plane; and (3) 10-mm resection from the lateral tibia plateau. Prior to registration of the bone landmarks, we made a small hole at each registration point using a 1.2-mm K-wire to minimize measurement error using computed tomography (CT). The small holes can be detected easily in CT images.
First, we measured the amount of bone resection using digital calipers (Monotaro, Hyogo, Japan) considering the thickness of the bone saw used in this study. The thickness of the bone saw was 1.27-mm. The resected proximal fragment including the tibia plateau was then attached to the main fragment of the tibia using 1.27-mm double-sided adhesive tape. The width of the adhesive tape was also the same as that of the bone saw. As the measurement of alignment using CT images is affected by the cutting plane, we referred to the standardized coordinate system using specifically designed measurement software (ZedKnee; LEXI, Tokyo, Japan) (Miura et al. 2018). We performed CT of the tibia and fibula pairs and entered the CT data in Digital Imaging and Communications in Medicine (DICOM) format into the ZedKnee software, which allows measurement of the angle of bone resection according to the coordinate axis defined by each investigator (Mochizuki et al. 2018). We defined the tibial long axis as a line connecting the tibial center on the tibia plateau and ankle center. The ankle center was defined as the point dividing the transmalleolar axis according to the ratio of 55% lateral to 45% medial (Kim et al. 2005). We defined the tibial anteroposterior axis as a line connecting the middle of the posterior cruciate ligament to the medial border of the patellar tendon attachment according to Akagi et al. (2004). Two investigators (HO and MN) independently measured varus/valgus angle in the coronal plane, posterior slope in the sagittal plane, and rotational alignment of the tibia.
We calculated the differences between the values displayed on the smartphone during use of the AR-KNEE system and measured values with CT or digital caliper.
This was a pilot study to assess the accuracy of tibial bone resection using the AR-KNEE system. We calculated the absolute values of the differences between angles measured using CT and angles displayed on the smartphone screen in terms of varus/valgus, posterior slope, and rotation angles. In addition, we calculated the absolute values of differences between the thickness of the resected tibia measured using digital calipers and target thickness of 10.0 mm. The averages, standard deviations, and interquartile ranges were calculated for each parameter. We compared the varus/valgus, posterior slope, and rotation angles displayed on the smartphone screen and the measurement values determined by CT using the Mann-Whitney U-test.
To test interobserver reliability, the intraclass correlation coefficients of postoperative measurements of bone resection angles were calculated for two assessors (HO and MN).
Table 1 summarizes differences between the values displayed on the smartphone screen and the actual measurement values in terms of varus/valgus angle, posterior slope angle, internal/external rotation angle, and the thickness of the resected bone. There were no significant differences between the angles displayed on the smartphone screen and the measurement angles determined using CT (P = 0.27, 0.06, and 0.10 in varus/valgus, posterior slope, and rotation angle, respectively).
The intraclass correlation coefficients between two assessors for varus/valgus, posterior slope, and rotation angles were 0.97 (95% CI, 0.92 to 0.99), 0.94 (95% CI, 0.81 to 0.98), and 0.79 (95% CI, 0.40 to 0.94), respectively.
The most important finding of this study was that the AR-KNEE system indicated varus/valgus and posterior slope angles of less than 1° and internal/external rotation angle < 2° for the differences between the values displayed on the smartphone screen and the actual measurement values with regard to bone resection of the proximal tibia using sawbones.
AR technology has been applied to various surgical procedures (Ogawa et al. 2018; Abe et al. 2013; Eftekhar 2016; Conrad et al. 2016). In TKA, one AR assisted system has been proposed in which the image of the bone exposed in the surgical field is intraoperatively projected to the image obtained by preoperative CT using an optical camera and computer (Pokhrel et al. 2019). Our AR-KNEE system is completely different from the system of Pokhel et al. (Pokhrel et al. 2019). The AR-KNEE system does not require preoperative CT and can superimpose tibial long and anteroposterior axes on the surgical field using smartphone and marker with two-dimensional bar codes. When the line does not fit the tibia properly, the surgeon can easily recognize that the registration is incorrect.
This pilot study was performed to assess the accuracy of the AR-KNEE system. The results indicated that the AR-KNEE system had accuracy comparable to other conventional navigation systems in terms of varus/valgus, posterior slope, and internal/external rotation angles (Stöckl et al. 2004; Pitto et al. 2006; Rosenberger et al. 2008). The mean absolute differences in previous studies ranged from 0.5° to 1.2° in varus/valgus angle and from 0.7° to 1.4° in posterior slope (Hasegawa et al. 2013; Tsukeoka et al. 2018; Feichtinger et al. 2018). Although the mean deference of 0.6 mm ± 0.7 mm in terms of the thickness of resected bone was also comparable to that of the conventional navigation system (Bäthis et al. 2005), we believe that the value of the mean difference can represent an unacceptable error for correct balancing during TKA. A recent cadaver study using robotically assisted technique showed more accurate results: the mean difference between planned bone resections and bone resections measured with calipers was 0.15 mm ± 1.08 mm at the lateral tibia plateau (Parratte et al. 2019). Therefore, at this time, we cannot recommend the AR-KNEE system as a means of reliable navigation for the thickness of the resected tibia.
This pilot study had several limitations. The materials used in this study were sawbones without soft tissue. The accuracy of registration touching bony landmarks through soft tissues was biased by the thickness of the soft tissue (Tsukada and Wakui 2010). This preclinical study cannot provide surgeons with information on whether the skin incision should be extended or not when using AR-KNEE in the clinical setting. Verification of the AR-KNEE system in patients is warranted.
All bone resections were performed by one surgeon in this study. The accuracy of bone resection may depend on the experience of the surgeon (Kazarian et al. 2019). Multi-surgeon studies will provide more robust external validity.
The AR-KNEE system may have a learning curve similar to other navigation systems for TKA (Jenny et al. 2008). It is possible that gaining experience with the AR-KNEE system would provide more accurate bone resection.
Applying the AR-KNEE system to bone resection of the proximal tibia, the absolute values of differences between the values displayed on the smartphone screen and the measurement values obtained using CT were 0.5° ± 0.2°, 0.8° ± 0.9°, and 1.8° ± 1.5° in varus/valgus, posterior slope, and internal/external rotation angles, respectively. The AR-KNEE system may become a useful alternative navigation system for TKA.
Availability of data and materials
The datasets used and/or analyzed during the current study are not publicly available. Data are however available from the corresponding author on reasonable request.
Digital Imaging and Communications in Medicine
Total Hip Arthroplasty
Total Knee Arthroplasty
Abe Y, Sato S, Kato K, Hyakumachi T, Yanagibashi Y, Ito M, Abumi K (2013) A novel 3D guidance system using augmented reality for percutaneous vertebroplasty: technical note. J Neurosurg Spine 19:492–501
Akagi M, Oh M, Nonaka T, Tsujimoto H, Asano T, Hamanishi C (2004) An anteroposterior axis of the tibia for total knee arthroplasty. Clin Orthop Relat Res 420:213–219
Bäthis H, Perlick L, Tingart M, Perlick C, Lüring C, Grifka J (2005) Intraoperative cutting errors in total knee arthroplasty. Arch Orthop Trauma Surg 125:16–20
Conrad C, Fusaglia M, Peterhans M, Lu H, Weber S, Gayet B (2016) Augmented reality navigation surgery facilitates laparoscopic rescue of failed portal vein embolization. J Am Coll Surg 223:e31–e34
de Steiger RN, Liu YL, Graves SE (2015) Computer navigation for total knee arthroplasty reduces revision rate for patients less than sixty-five years of age. J Bone Joint Surg Am 97:635–642
Eftekhar B (2016) App-assisted external ventricular drain insertion. J Neurosurg 125:754–758
Feichtinger X, Kolbitsch P, Kocijan R, Baierl A, Giurea A (2018) How accurate is intraoperative alignment measurement with a navigation system in primary total knee arthroplasty? J Knee Surg 31:467–471
Hasegawa M, Yoshida K, Wakabayashi H, Sudo A (2013) Cutting and implanting errors in minimally invasive total knee arthroplasty using a navigation system. Int Orthop 37:27–30
Jenny JY, Miehlke RK, Giurea A (2008) Learning curve in navigated total knee replacement. A multi-centre study comparing experienced and beginner centres. Knee 15:80–84
Kazarian GS, Lawrie CM, Barrack TN, Donaldson MJ, Miller GM, Haddad FS, Barrack RL (2019) The impact of surgeon volume and training status on implant alignment in total knee arthroplasty. J Bone Joint Surg Am 101:1713–1723
Kim SJ, MacDonald M, Hernandez J, Wixson RL (2005) Computer assisted navigation in total knee arthroplasty: improved coronal alignment. J Arthroplast 20(Suppl):123–131
Miura M, Hagiwara S, Nakamura J, Wako Y, Kawarai Y, Ohtori S (2018) Interobserver and intraobserver reliability of computed tomography-based three-dimensional preoperative planning for primary total knee arthroplasty. J Arthroplast 33:1572–1578
Mochizuki T, Tanifuji O, Koga Y, Sato T, Kobayashi K, Watanabe S, Fujii T, Yamagiwa H, Katsumi R, Koga H, Omori G, Endo N (2018) Correlation between posterior tibial slope and sagittal alignment under weight-bearing conditions in osteoarthritic knees. PLoS One 13:e0202488
Novak EJ, Silverstein MD, Bozic KJ (2007) The cost-effectiveness of computer-assisted navigation in total knee arthroplasty. J Bone Joint Surg Am 89:2389–2397
Ogawa H, Hasegawa S, Tsukada S, Matsubara M (2018) A pilot study of augmented reality technology applied to the acetabular cup placement during total hip arthroplasty. J Arthroplast 33:1833–1837
Parratte S, Price AJ, Jeys LM, Jackson WF, Clarke HD (2019) Accuracy of a new robotically assisted technique for total knee arthroplasty: a cadaveric study. J Arthroplast. https://doi.org/10.1016/j.arth.2019.06.040
Pitto RP, Graydon AJ, Bradley L, Malak SF, Walker CG, Anderson IA (2006) Accuracy of a computer-assisted navigation system for total knee replacement. J Bone Joint Surg Br 88:601–605
Pokhrel S, Alsadoon A, Prasad PWC, Paul M (2019) A novel augmented reality (AR) scheme for knee replacement surgery by considering cutting error accuracy. Int J Med Robot 15:e1958
Rosenberger RE, Hoser C, Quirbach S, Attal R, Hennerbichler A, Fink C (2008) Improved accuracy of component alignment with the implementation of image-free navigation in total knee arthroplasty. Knee Surg Sports Traumatol Arthrosc 16:249–257
Slover JD, Tosteson AN, Bozic KJ, Rubash HE, Malchau H (2008) Impact of hospital volume on the economic value of computer navigation for total knee replacement. J Bone Joint Surg Am 90:1492–1500
Stiehl JB (2007) Computer navigation in primary total knee arthroplasty. J Knee Surg 20:158–164
Stöckl B, Nogler M, Rosiek R, Fischer M, Krismer M, Kessler O (2004) Navigation improves accuracy of rotational alignment in total knee arthroplasty. Clin Orthop Relat Res 426:180–186
Tsukada S, Wakui M (2010) Decreased accuracy of acetabular cup placement for imageless navigation in obese patients. J Orthop Sci 15:758–763
Tsukeoka T, Tsuneizumi Y, Yoshino K, Suzuki M (2018) Case-related factors affecting cutting errors of the proximal tibia in total knee arthroplasty assessed by computer navigation. Knee Surg Sports Traumatol Arthrosc 26:1493–1499
There was no external source of funding for this study.
The study protocol was approved by the institutional review board of Hokusuikai Kinen Hospital.
Consent for publication
The publication of this study was approved by the institutional review board of Hokusuikai Kinen Hospital.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Tsukada, S., Ogawa, H., Nishino, M. et al. Augmented reality-based navigation system applied to tibial bone resection in total knee arthroplasty. J EXP ORTOP 6, 44 (2019). https://doi.org/10.1186/s40634-019-0212-6
- Computer assisted surgery
- Virtual reality