Innovation And Application Of Augmented Reality-Based Minimally Invasive Spine Surgery Guiding System

BackgroundDigitalization,minimal invasion and intelligence is a significant trend of surgery and the important sign of advanced surgery technique.Minimally invasive spine surgery(MISS)is one that by virtue of the extent and means of surgical technique results in less collateral tissue damage,resulting in a measurable decrease in morbidity and more rapid functional recovery than traditional exposures,without differentiation in the intended surgical goal.MISS procedures reduce blood loss,decrease postoperative back pain,and shorten the hospital length of stay compared to those with traditional open procedures.After decades of development,MISS has become widespread globally.However,a significant learning curve is also involved in attaining proficiency with MISS procedures and instrumentations,and complication rates are higher during this learning period.Surgical navigation technology,rapid prototyping technology,surgical robotics technology,X-Reality technology,etc.,are several major branches in the field of digital minimally invasive spine surgery,among which X-Reality technology is one of the most popular scientific research directions at present.X-Reality technology,is a long-term,with the concept of inclusive and extensibility,the floorboard of the all kinds of virtual and real world interactive technology,including now of virtual reality technology,augmented reality,mixed reality technology and cinematic reality technology,as well as the future of other new interactive technology of virtual and real world.Augmented reality(AR)is an interactive experience simulating a real-world environment wherein the objects are “augmented” by computer-generated perceptual information.With the help of advanced AR technologies,such as computer vision and object recognition,information on the surrounding real world becomes interactive and manipulatable by the user.Information on the environment and its objects is overlaid onto the real world.AR is a significant technique in complicated surgery procedures,especially when it’s using for guiding procedures,it’s similar as navigation system.Percutaneous pedicle screw technique is a routine operation in MISS procedures,which makes minimally invasive fixation of spinal vertebrates with less muscular lesion,less blood loss,less postop pain and shorter operation time and hospital stay,better for rehabilitation.However,the accuracy of pedicle screw placement is highly dependent on the guidance of the intraoperative fluoroscopy,which takes longer time than in traditional open surgery.We aimed to develop an innovative AR-based MISS guiding system and to assess the feasibility and accuracy of percutaneous lumbar pedicle screw placement using AR-based MISS guiding system in experimental settings.Materials and MethodsPart ⅠWe used Microsoft HoloLens as AR device to develop AR-based MISS guiding system.Development software as Microsoft Visual Studio Community 2017,Materialise Mimics 20.0,Materialise 3-Matic 12.0,Autodesk 3DSMAX 2017,Adobe Photoshop CC and Unity Personal with Vuforia Augmented Reality Support was deployed into an X86-architecture PC.Registered as Vuforia Augmented Reality developer on the website of Vuforia Developer Portal and got develop license key for Vuforia.Defined a test database in Target Manager.Created a black white asymmetric high contrasted cubic picture.Added the picture into database as a Single Image Target named test.Created a Vuforia Augmented Reality scene in Unity and imported Vuforia database,added text as “Hello World” attached to Image Target.Built C# project in Unity and Visual Studio and deployed onto Microsoft HoloLens.Printed picture of Image Target named test,launched the deployed application in Microsoft HoloLens and gazed at printed picture.Part ⅡA human-lumbar-like sawbone was placed into a plastic container.Sawbone was fixed with K-wire.An agar solution with a concentration of 6% was prepared,heated to dissolve completely,poured into the container and cooled naturally to solidify.A cube of appropriate size was created in Autodesk 3DSMAX and exported into a Standard Tessellation Language(STL)file.The STL file were converted into G-Code in Ultimaker Cura.The cubic model was 3D printed with a Vuforia Image Target pasted onto it.K-wires for fixation were recycled.The phantom was inverted on the retaining plate and fixed with K-wires.Cubic marker with Vuforia Image Target was fixed onto the phantom with K-wires.A Computer Tomography(CT)scan of the phantom was acquired and the Digital Imaging and Communications in Medicine(DICOM)data was recorded into disc.DICOMDIR file was imported into Materialise Mimics to build 3D masks of each part of the phantom.The Crop Mask function was used for rough cropping and the Edit Masks erasing Lasso tool for fine clipping.3D model of each part was acquired with Calculate Part tool.After smoothing,each model was remeshed in Materialise 3-Matic and exported into STL file which was converted into Autodesk FBX file in Autodesk 3DSMAX for later use.A Vuforia Augmented Reality scene was created with FBX file and Vuforia Image Target database imported.A C# project was built and deployed into Microsoft HoloLens.The operator launched the application on Microsoft HoloLens and gazed at the phantom.The inner structure of the phantom would be displayed.The operator placed 10 K-wires into bilateral pedicles of 5 lumbar vertebrates.And a CT scan was acquired after all placements.Result would be analyzed on image diagnosis station.Placement of each K-wire would be counted and classified as K-wire completely inside the pedicle,K-wire around the outer wall of pedicle,K-wire around the inner wall of pedicle,K-wire perforate or miss the pedicle.All placement would be considered safe except K-wire perforate or miss the pedicle.Part ⅢTwelve human-lumbar-like phantoms were built as before.Cubic marker with Vuforia Image Target was fixed onto the phantom with K-wires.A CT scan of the phantom was acquired and the DICOM data was recorded into disc and imported into AR develop station.A STL file was exported after modeling,post process and trimming.Model was imported into Unity and built into C# project and deployed into Microsoft HoloLens.The operator launched the application on Microsoft HoloLens and gazed at the phantom.The inner structure of the phantom would be displayed.Twelve phantoms were divided into 2 groups(AR and X-Ray).The AR group was further divided into 2 subgroups(Automatic-AR,Manual-AR)by hologram aligning methods.Two experienced spine surgeons independently performed 20 AR-guided percutaneous lumbar pedicle screw placements with automatic alignment,20 AR-guided percutaneous lumbar pedicle screw placements with manual alignment,and 20 radiograph-guided percutaneous lumbar pedicle screw placements.The operators targeted the bone surface using a percutaneous lumbar puncture needle and attempted to place a K-wire into the pedicle to simulate pedicle screw placement.CT scan of each phantom was performed after all K-wire placements.For each K-wire,the insertion path was expanded to a 6.5-mm diameter to simulate a lumbar pedicle screw used in real surgical practice.The operative time required for each placement from needle setup to finishing K-wire placement was recorded.Two independent radiologists rated all images of K-wire placements in a randomized order.The radiologist was blinded to the guiding method and experimental group.Outcomes were classified as grade I(no pedicle perforation),grade II(screw perforation of the cortex by up to 2 mm),and grade III(screw perforation of the cortex by > 2 mm).In a clinical situation,placements scored as grade I or II would be acceptable and safe for patients.All data was analyzed with statistics software.ResultPart ⅠThe develop station and develop software was successfully deployed.After launching deployed application on Microsoft HoloLens,operator gazed at the printed Image Target named test,the text “Hello World” was displayed.The character changed the display angle as the viewing angle changed.The text disappeared when the view was taken away from the image.When the pattern appeared again in the field of view,the text appeared again,and the position and Angle exactly match the position and Angle of the pattern.Part ⅡA phantom was built successfully and compatible for digitalization and post process.The digital phantom was used to build the automatic aligning function of AR-based MISS guiding system.When operator launched the application and gazed at the phantom,the inner structure of the phantom was displayed.The structure changed the display angle as the viewing angle changed.An operator performed 10 K-wire placements by automatic aligning AR guiding methods.The classification after CT scan was acquired: K-wire completely inside the pedicle(n=5),K-wire around the outer wall of pedicle(n=1),K-wire around the inner wall of pedicle(n=2),K-wire perforate or miss the pedicle(n=2).8 of 10 placements were in compliance with safety standards.Part ⅢTwo operators performed 40 pedicle screw placements simulated by K-wire placement by two different hologram aligning methods,and 20 pedicle screw placements simulated by K-wire placement under X-ray guidance.Chi-square test was performed for the accuracy of the placement guided by the two methods,P=0.000<0.05,the difference was statistically significant.Chi-square test was performed for the safety of the placement guided by the two methods,P=0.106>0.05,the difference was not statistically significant.Two operators performed 40 pedicle screw placements simulated by K-wire placement by two different hologram aligning methods.Chi-square test was performed for the accuracy of the placement guided by the two hologram aligning methods,P=0.526>0.05,the difference was not statistically significant.Chi-square test was performed for the safety of the placement guided by the two hologram aligning methods,P=0.644>0.05,the difference was not statistically significant.Wilcoxon rank-sum test was used to examine the differences in the time taken for pedicle screw placement under two different aligning methods,P=0.0081<0.05,the difference was statistically significant.Wilcoxon rank-sum test was used to examine the differences in the time taken for pedicle screw placement under two different guiding methods,P=0.0000<0.05,the difference was statistically significant.Conclusion1.It was feasible to use Microsoft HoloLens as the technology carrier of AR-based MISS guiding system.2.In agar model experiment settings,percutaneous pedicle screw placement guided by AR-based MISS guiding system was feasible and safe.3.In agar model experiment settings,AR-based MISS guiding system with automatic aligning method was with higher efficiency than AR-based MISS guiding system with manual aligning method.