Photogrammetry Applied to Neurosurgery: A Literature Review

Photogrammetry refers to the process of creating 3D models and taking measurements through the use of photographs. Photogrammetry has many applications in neurosurgery, such as creating 3D anatomical models and diagnosing and evaluating head shape and posture deformities. This review aims to summarize the uses of the technique in the neurosurgical practice and showcase the systems and software required for its implementation. A literature review was done in the online database PubMed. Papers were searched using the keywords “photogrammetry”, “neurosurgery”, “neuroanatomy”, “craniosynostosis” and “scoliosis”. The identified articles were later put through primary (abstracts and titles) and secondary (full text) screening for eligibility for inclusion. In total, 86 articles were included in the review from 315 papers identified. The review showed that the main uses of photogrammetry in the field of neurosurgery are related to the creation of 3D models of complex neuroanatomical structures and surgical approaches, accompanied by the uses for diagnosis and evaluation of patients with structural deformities of the head and trunk, such as craniosynostosis and scoliosis. Additionally, three instances of photogrammetry applied for more specific aims, namely, cervical spine surgery, skull-base surgery, and radiosurgery, were identified. Information was extracted on the software and systems used to execute the method. With the development of the photogrammetric method, it has become possible to create accurate 3D models of physical objects and analyze images with dedicated software. In the neurosurgical setting, this has translated into the creation of anatomical teaching models and surgical 3D models as well as the evaluation of head and spine deformities. Through those applications, the method has the potential to facilitate the education of residents and medical students and the diagnosis of patient pathologies.


Introduction And Background
The word "photogrammetry" describes the process of creating three-dimensional (3D) models from standard two-dimensional (2D) photography with measurable quantitative data [1,2].
In the neurosurgical field, the applications of photogrammetry correlate with the technique's main functions -3D model creation, mainly used for the visualization of neurosurgical anatomy  and for skeletal and bony deformity evaluation .
The process normally involves a series of steps, with the first one being the acquisition of the images from different photographic angles.These images then go through a process of orientation, triangulation (calculating the position of the point or object in 3D), point-cloud generation (identifying common points among the photographs), surface reconstruction (triangulation by using the point cloud), and texture mapping, where the 3D model is finalized [84].Nowadays these steps are done by using dedicated software [1,2].
In the neuroanatomy laboratory, photogrammetry is used to generate 3D models based on cadaveric dissections [2,18,19], which allows for their repeated use and their prolonged preservation in the virtual space and their application in residents training to acquire a better understanding of complex anatomy.The purpose of the models ranges from a more general representation of anatomy [5,7,8,19,21] to the creation of collections of specific representations of surgical approaches [13][14][15][16]20,22].
The method is gradually finding its way into clinical practice with the diagnosis of craniosynostosis and non-craniosynostosis skull deformities  and scoliosis .This is because photogrammetry offers the possibility to estimate the shape of objects and their distance from one another.Thus, numerous researchers have started exploring its use in conditions that present themselves with superficial misalignments of body composition.The devices and software used for this purpose are constantly evolving with the advancement of technology.
Additionally, more specific cases of the use of photogrammetry in the field of neurosurgery are appearing, such as in cervical-spine surgery [85], skull-base surgery [86], and radiosurgery [87].
The purpose of this review is to summarize the uses of photogrammetry in neurosurgical practice.Furthermore, we offer insight into the systems and software required for the implementation of these techniques in everyday practice.

Literature Review
To present the uses of photogrammetry in the neurosurgical field, a narrative review was conducted on the online database PubMed.The keywords used were "photogrammetry", "neurosurgery", "neuroanatomy", "craniosynostosis" and "scoliosis".We have not limited the search results to a certain year.Nevertheless, most of the articles in the "Neuroanatomy, Surgical Anatomy" section were published in the last four years and the majority of the articles in the other sections were published after the year 2010.

Inclusion and Exclusion Criteria
The main inclusion criteria of the article correlate with the paper's aim -to provide concise information on the uses of photogrammetry in neurosurgical education, teaching, evaluation, and diagnosis of patients.Therefore, the following inclusion criteria were used: (1) photogrammetry was the main method used in the study (model creation, patient evaluation); (2) the systems and software used for the photogrammetric technique are specified in the paper; (3) paper is written in English; and (4) the technique was applied to the field of neurosurgery.The main exclusion criteria were as follows: (1) non-photogrammetry studies; (2) studies in which photogrammetry was used for another field of medicine (3) papers not available in full text; (4) animal studies; and (5) papers that did not specify what systems and software were used for the study, therefore, the technical part of the photogrammetry process could not be presented.
The papers that did not meet the criteria were excluded.The remaining studies were included in the review.We have created a flowchart that summarizes the process of screening (Figure 1).

FIGURE 1: Narrative Review Flowchart
Flowchart presenting the work process behind the review -the different stages of identification and screening of articles.

Results
After the primary and secondary screening were conducted and the additionally identified articles included, we were left with a total of 86 studies out of the 315 initially identified.A great number of studies among the "unrelated articles" were excluded because they did not fully encompass the subject in question.This would mean that they either did not use photogrammetry, even though they were related to neurosurgery, or they  The articles were classified into three main groups according to their scope: Group 1 -"Neuroanatomy, Surgical Anatomy", Group 2 -"Craniosynostosis, Non-synostotic Cranial Deformities, and Skull Development", and Group 3 -"Scoliosis and Posture".

Neuroanatomy, Surgical Anatomy
In the first group, we've included the papers that showed the use of photogrammetry in neuroanatomy for the creation of 3D anatomical models .All of the papers used cadavers for the creation of the models, except one, which explored the generation of intraoperative 3D models using Digital Single-Lens Reflex (DSLR) cameras [11].For image acquisition, in the studies, the authors used DSLR cameras [3,8,9,[11][12][13][14][15][16]18,19,21,22] smartphone cameras [1,2,[4][5][6][7][8]20], and endoscope cameras [10,17].The consensus among the studies is that anatomical models of high quality and accuracy can be created using ordinary cameras and smartphones.For the image acquisition itself, the authors recommend that the parts in shade are lightened, and the lit areas darkened so that the quality and clearance of the model are similar all around [18].
For the creation of the model from 2D pictures, different types of software exist and they usually require the use of many images showing different points of view of the object, such as Autodesk ReCap Photo (Autodesk Inc., San Francisco, CA, USA) [11,17], Reality Capture Beta (Capturing Reality, Bratislava, Slovakia) [18], and Agisoft Metashape software (Agisoft LLC, St. Petersburg, Russia) [19].Software solutions for smartphone-based photogrammetry, such as Metascan (Abound Labs Inc., New York, NY, USA) [19] or Qlone (QloneC 2017-2020, EyeCue Vision Technologies Ltd, Yokneam, Israel) [6] have allowed moving the processing phase from the device to a cloud space which further facilitates the method.Taking the process a step further, the AI-integrated Intel ISL MiDaS v2.1 (Intel Labs, Santa Clara, CA, USA) software is capable of monocular depth estimation derived from the color intensity of every specific pixel.This creates the opportunity for 3D model creation from a single 2D photo, which according to the study could potentially close the distance between neuroimaging and photogrammetry scanning technologies [3,4,9].
For the editing of the 3D model, the latter is uploaded on a special 3D editing software, such as MeshLab (Visual Computing Lab, ISTI-CNR, Pisa, Italy) [2,3,9], Autodesk Meshmixer (https://meshmixer.com/)[18] or Blender (Blender Foundation, Amsterdam, The Netherlands) [10,11,19,20].This step is necessary for the quality improvement of the models, which usually includes the removal of unnecessary elements [9,18,19].Another innovative aspect of the 3D models is that they can be used in virtual reality (VR) and augmented reality (AR) environments [1,6].With dedicated software, such as the Qlone software the model can be visualised in an AR environment.The model needs to be placed on a surface, such as a table.Furthermore, the model can be uploaded on a VR headset [1].These advancements make the process more interactive.
The methodology for the creation of intraoperative 3D models is similar to the one used in the cadaver model creation [11].In the study, the ReCap Pro software was used for the model creation, and Blender software for improving the model.The result is the creation of a high-quality model realistically visualizing the operative field.

Craniosynostosis, Non-synostotic Cranial Deformities, and Skull Development
The diagnosis of craniosynostosis through photogrammetry is based primarily on shape recognition.Both expensive equipment [25,32] and smartphone cameras [23] can be used for the image acquisition part of the process.Additionally, a deep-learning algorithm that uses a database of pre-uploaded head images, normal and pathological, can speed up the process [24].Larger datasets of patients with different head shapes are necessary for the training of the algorithm's learning capabilities, which will potentially be facilitated through the use of DSLR cameras for the image-capturing part of the process.Several papers in this section presented the use of photogrammetry in postoperative evaluation [25,26,29,33,36,44,50] which, if applied widely, can potentially reduce radiation exposure during patient follow-up, whilst maintaining objectivity.However, two studies compared the method with CT scans and reported it as a valuable alternative, not as superior [29,50].
A review article that explored the topic of photogrammetry in craniosynostosis published in the year 2022 has been identified [28].Our review confirms the results of the paper and goes more in-depth on the software and systems used for the method.Additionally, we identified four papers published after the submission date of the aforementioned review [27,31,34,47].Notably, among these studies, Kronig et al.
propose a novel approach to diagnosing craniosynostosis through photogrammetry with the use of the Utrecht Cranial Shape Quantificator method, which was successful in diagnosing each of the five types of craniosynostosis [31].Shaufelberger et al. employed the creation of a statistical shape model of a craniosynostosis head from a large dataset of patients which demonstrates results comparable to the CT scans [47].
The most widely used systems among the studies are the 3dMD System (3dMD Inc., Atlanta, GA, USA) [24,25,30,42], and the Canfield VECTRA-360 nine-pod system (Canfield Scientific, Parsippany, NJ, USA) [27,47,49,51], which are developed specifically for the creation of 3D models from images.The systems come with proprietary software that deals with the integration of the images into one single model.For the refinement of the model third-party software is used, such as Blender [30,42] and MeshLab [27,47].The alternative to using such technology is the use of standard cameras and third-party software both for the creation of the model and for its refinement, which was described in one paper [23].The method removes the need for expensive equipment and potentially incentivizes more specialists to apply it through the comfort of their mobile phones.According to studies, photogrammetry is a valuable tool for craniosynostosis evaluation and follow-up, which offers comparable results to CT without excessive radiation [37,49].The use of photogrammetry, however, cannot yet fully replace CT scans as it has demonstrated limited diagnosis capabilities in milder cases of the condition [30].
When evaluating non-synostotic cranial deformities, which are becoming more prevalent with the infants' supine position (positional plagiocephaly), commonly chosen to prevent sudden infant death syndrome), photogrammetry demonstrated good results for the diagnosis, treatment planning, and follow-up of the patients [45,46].Any deviations in the process of skull development can lead to esthetical and psychological problems later in life.By making measurements through photogrammetry at two different points in time the development of the skull can be assessed and any deviation from the normal growth can be detected early on [27,39,40].

Scoliosis and Posture
Photogrammetry has been used for the detection of scoliosis for a couple of decades now.The first instances involve the use of the "Moiré topography", a method utilizing the projection of a series of light stripes over the patient's back [64,65,74,82,83].Later the methods evolved through the use of "Rasterstereography", which involves the use of a special system -the Formetric system (DIERS International Gmbh, Schlangenbad, Germany), to capture a series of pictures of the back of the patient from different angles, which are later analyzed by dedicated software [55,63,67,78].Nowadays technology has progressed to allow the use of DSLR cameras for the purpose of image acquisition along with dedicated software to analyze the data [57,58,67,69].
When comparing the photogrammetric method with the radiographic method, it reflected accurately the progression of lateral vertebral deviation and vertebral rotation.[78,80].The method cannot replace radiography, however, it can potentially be applied in the follow-up of patients to dilute the number of radiographs to once every 12 to 18 months, provided the photogrammetric evaluation does not show rapid worsening of scoliosis.The method has been applied as a non-ionizing means of school screening of children for scoliosis [58,61].The main limitations of photogrammetry in that aspect arise with individuals of high Body Mass Index, due to the greater challenge in localizing the spinous processes.Furthermore, the surgical removal of the spinous processes renders the method unusable, since the anatomical structure used to make the measurement is absent in these patients [58].
Photogrammetry is also a reliable method to evaluate different parameters of the human posture, such as the trunk length and inclination, as well as the kyphotic and lordotic angles [68,71,72].The use of machine learning algorithms could hypothetically facilitate even further the assessment and bring it to the level of comfort of a mobile application [69].Through the use of an AI-powered application, with the automatic identification and analysis of "Anatomical and Segment Points", points correlating with anatomical sites, such as the trochanters, anterior superior iliac spine, and the acromion, the detection of postural abnormalities, joint/muscle impediments, range of motion, and the monitoring of patient rehabilitation could be assisted.Nevertheless, further studies are needed for the smoother implementation of such an application in clinical practice.

Other Applications
We have identified three distinct papers showcasing the use of photogrammetry for range of motion assessment in cervical spine surgery, skull-base surgery, and radiosurgery.
In cervical spine surgery, laminoplasty is a procedure used to treat multilevel spine stenosis and is associated with a change in the range of motion of the spine postoperatively.Photogrammetry can be used as an inexpensive and radiation-free method to measure the exact extent of this change in all planes [85].
The study used pre-and postoperative images taken by DSLR camera.The chin-brow vertical angle was measured during flexion and extension for the sagittal plane and the nose-turn angle was used for the axial plane.In skull-base surgery, one of the major challenges is the closure of dural defects.This defect is usually fixed either through mobilizing the pericranium (autograft) or by using artificial implants.The size of the material is usually subjectively assessed by the operator.Through the use of photogrammetry, the size of the implant can be accurately calculated using contour and vertex color to determine the size of the defect [86].In radiosurgery, photogrammetry has been used to assess the effect on the outcome of the movement of the patient's head during the procedure [87].The VisionRT (VisionRT Ltd., London, UK) system was used for the purpose.The study concluded that treatment plans that target multiple metastases by using the same point of radiation are more sensitive to patient movement.

Discussion
In this paper, we have evaluated the application of a relatively novel (for neurosurgery) algorithm for surface scanning, photogrammetry, and its various applications to neurosurgical practice, including mainly neurosurgical anatomy but also various potential clinical applications, such as cranial deformity and scoliosis evaluation, where the method can potentially reduce radiation exposure.

Neurosurgical Anatomy Training
Studying the complex anatomy of the nervous system is a long and difficult process.Answering the need to diversify the methods for this purpose, photogrammetry has been applied for the creation of accurate 3D models of different anatomical components.
The studies in this review show that both general neuroanatomy models and specific neurosurgical models and simulations of approaches can be created with the photogrammetry method .
The development of 3D models in neuroanatomical teaching can potentially revolutionize the field since it offers a solution to many of the problems that standard teaching methods are facing.The methods offer the possibility to manipulate the model by zooming in and out of regions of interest to study the specific, often annotated structures [1,2,20].Through this, the examiner can acquire knowledge about the actual proportions and depth of a specific model, the distance between different structures, and their respective positions in relation to other structures.This gives the advantage of the 3D models when compared with the conventional 2D images used for neuroanatomy teaching.Both medical students and neurosurgery residents can use the models to learn the complex anatomy of the nervous system and spatial orientation during cranial surgery [17].
The introduction of the models in VR and AR environments makes the process more interactive.Through AR, the user can project the models on any surface, enlarge them, and look around every detail of the model [1,6].With a VR headset, the user can create a virtual neuroanatomy lab [1,4,17] and examine the models with greater ease.The VR device offers the possibility to grab the model and manipulate it in every plane, thus facilitating the process of examination.
Furthermore, many of the logistical difficulties associated with acquiring cadavers for neuroanatomy training, such as the cost and the ethical concerns are in great part dealt with through the use of the 3D models.The lack of cadavers in neuroanatomy laboratories makes the easy distribution of the models an even more valuable trait.With time the deterioration of the models and the difficulties associated with their conservation is another burden for current anatomy departments.When a 3D model is uploaded on a platform it can remain there for an indefinite period of time, thus the conservation of a valuable anatomical model and a potentially rare anatomical variant is facilitated through the use of the photogrammetric technique.
The studies show the increasing use of photogrammetry in neurosurgical anatomy as it provides authenticity and photorealism to 3D models [1,2].The resulting 3D photorealistic models can be used for anatomical and neurosurgical training in augmented (Figures 2, 3) and virtual reality (Figure 4).When a 3D model is uploaded on a platform it can remain there for an indefinite period of time, thus the conservation of a valuable anatomical model and a potentially rare anatomical variant is facilitated through the use of the photogrammetric technique.Published with the permission of the "3D Atlas of Neurological Surgery" platform [88].Published with the permission of the "3D Atlas of Neurological Surgery" platform [88].
Through the application of the photogrammetric method, a collection of immersive surgical anatomy of different neurosurgical approaches has been already published in the literature [10,[13][14][15][16]18,21,22].There are very detailed papers for cadaver-based models for complex approaches such as Far-Lateral [13], Frontotemporal-Orbitozygomatic [14], Pterional [15], Retrosigmoid [16], and the Smith-Robinson approach to the anterior spine [22].Despite that, cadaver training remains essential in neurosurgery resident's education and the new technologies can be regarded as a supplement to traditional methods of teaching and training [89].Therefore photogrammetry-based models and a new training curriculum in VR can be a good supplement to traditional training methods [4].In their studies, Aydin et al. and Gonzales-Romo et al. have created virtual neuroanatomy laboratories where the models are uploaded and examined [1,4].According to the authors, before the application of photogrammetry, the only available option for studying neuroanatomy in AR and VR was through models produced by images and drawings in computer programs, which are limited by the artist's capability and anatomical knowledge.Nowadays, with the creation of 3D models from cadaver dissections, the method is becoming more accurate since it no longer depends on a non-specialized point of view of the captured field.The reported limitations are loss of image quality and long durations of the creation process of the models.
Combining photogrammetry anatomical scans with CT or MRI-segmented 3D data is another innovation that can potentially improve neurosurgical training.Hanalioglu et al. used an integrated method between 3D reconstructions of MRI and CT scans, neuronavigation, and photogrammetry to create quantitatively and qualitatively accurate 3D models of the pterional craniotomy [9].The model allows for a layer-by-layer simulation of the technique.However, one of the drawbacks of the method is that cadaver neuroimaging doesn't allow for visualization of the vessels, thus they had to be rendered with the help of the MeshLab Software.Another study by Roh et al. used photogrammetry brain scans combined with a CT-based 3D model of a skull to simulate neurosurgical approaches in a VR environment [17].Despite the obvious advantages of the method, there was no tactile feedback or the possibility to simulate surgical instruments.All these studies concerning neurosurgical anatomy may revolutionize the way anatomy is taught in the next few years and potentially improve the quality of education in places where cadaver training is limited.

Clinical Application
Two distinct pathologies concerning neurosurgery, characterized by deformation of the surface details of body parts are craniosynostosis  and scoliosis  are currently studied by photogrammetry.There is growing data that photogrammetry can be used to diagnose, follow up, and screen these conditions.This is possible since both craniosynostosis and scoliosis are conditions, associated with surface deformation of the shape of the head and spine respectively.Furthermore, the current standards for diagnosing and following these conditions are largely associated with radiation exposure to the patients (predominantly children).When applied with the right technique and software, photogrammetry offers the possibility of shape evaluation without the excessive radiation burden [38,57,58].
Generally, when managing craniosynostosis, anthropometric measurements are used for pre-and postoperative evaluation.However, they are time-consuming and rely on the physician's expertise which is subjective and largely depends on experience.A more objective method is the use of CT to confirm the diagnosis.CT however is associated with radiation exposure.
Most of the studies described the use of expensive systems for the acquisition of images [24,25,47,49].One study, however, described the possibility of replacing expensive equipment with ordinary smartphone technology [23].The authors explain that using a DSLR camera has the limitations of appropriate lighting combined with a static patient, which is difficult in the pediatric patient population unless anesthesia is applied.Using a standard smartphone camera for the acquisition of slow-motion videos was the method of choice, which provided a decent number of suitable images of different angles, necessary for the creation of the 3D model.A cap with stickers working as targets was also put on the infant's head, on one hand, to prevent the hair from getting in the way of the model creation and on the other because the targets facilitate the model creation.Overall, the image acquisition required three to five minutes without the use of any special equipment, considering the fact that the patient was held in place by an adult.
No study demonstrated a comparison of the different camera systems used for the acquisition of the images, thus no consensus has so far been reached on which is the most suitable one.Future research should be directed toward a comparison of the different imaging systems and software for the evaluations of cranial deformities.
The diagnosis and evaluation of scoliosis and its severity are crucial steps in the treatment strategy of the condition.The method of choice is usually radiography, through which the Cobb angle is measured.It gives insight into the progression of the condition and helps in planning the intervention if needed.The drawback of the method is the radiation exposure of the patients.The use of photogrammetry in scoliosis diagnosis relies on the measurement of the angles between various points of the spine, such as the angles between the vertebral bodies and the angles between the vertebral body and the horizontal plane [70].Differences in the height of the shoulders and the prominence of the waist are also observed.
Another method that precludes radiation exposure is the use of a scoliometer that can accurately measure the angle of trunk rotation in scoliosis patients.Photogrammetry has been compared with the scoliometer for its ability to measure the angle of trunk rotation when applied correctly [70].The authors used a DSLR camera and the so-called DIPA (Digital Image-based Postural Assessment) software (The Mathworks Inc., Natick, MA, USA), which is a dedicated software for the analysis of posture on images.After acquiring the data and comparing it with the scoliometer measurements, the two methods showed a high correlation with an error of 3° and a significance level of P ≤ .05.Thus, the method can be adequately used for the diagnosis of mild to severe scoliosis and with the added benefit of preserving a photograph of the patient with the applied measurements for future reference if needed.However, according to the data and the protocols that are published up until now, the method cannot replace the radiography, it can only be used as a supplement to it.Further improvement is needed before the complete replacement of radiography by surface scanning is deemed adequate.
With the advancement of the method, numerous other uses of photogrammetry are appearing, such as in cervical-spine surgery after laminoplasty [85], in skull-base surgery for repairment of the dural defect [86], as well as in radiosurgery intraoperatively [87].
All these applications of the technology in the neurosurgical field show its potential and the evolution of the photogrammetric technique used for these purposes has been steady over the past few years.Starting from expensive and complex devices [30,81] it has now arrived at the point where a regular DSLR camera [3,18,19,22] or even a smartphone [6,20,23,69,90] with cloud computing and dedicated software can be used to accomplish the tasks in minutes.The major limitation of our study is the use of a singular database -PubMed.Although PubMed has been widely regarded as one of the most comprehensive biomedical literature sources, a systematic review featuring a larger number of databases and registers could potentially offer a wider perspective of the subject.

Conclusions
The results of this review demonstrate that the application of photogrammetry in neurosurgery is promising.The method is used in the educational setting for the creation of normal and surgical anatomy 3D models.The models show high accuracy and correlation with the anatomy of interest, are also relatively simple to create, and can be uploaded to VR and AR environments, thus offering a new mode of education for residents and medical students.
Furthermore, in the clinical setting, photogrammetry offers a non-invasive and non-ionizing method for the diagnosis and evaluation of conditions associated with head and spine deformation.Through the acquisition of images and their analysis with dedicated software, a diagnosis and assessment value in most aspects similar to that of radiographs and CT scans can be achieved, without the excessive radiation burden.Nevertheless, the method requires further improvements and research, particularly associated with its use in clinical applications.

FIGURE 2 :
FIGURE 2: Screenshot of a direct comparison of real anatomical dissection of posterior neck region (A) and augmented reality model (B) placed next to each other to demonstrate the level of photorealism of the 3D model.3D -Three-dimensional; L -left; R -Right; S -Superior; I -Inferior The scale bar is in centimeters (upper left corner).

FIGURE 3 :
FIGURE 3: Photo demonstrating the real view of the observed augmented reality settings in the anatomical laboratory.3D-Three-dimensionalPublished with the permission of the "3D Atlas of Neurological Surgery" platform[88].

Result after image processing Image acquisition Image analysis software Post-processing software References
Agisoft Metashape software (Agisoft LLC, St. Petersburg, Russia); Autodesk ReCap Photo (Autodesk, San Rafael, California, USA);

TABLE 1 : Photogrammetry Table The
table presents the systems and software identified in the four groups of studies -Neuroanatomy, Surgical Anatomy; Craniosynostosis, Skull-Shape and Development; Scoliosis and Posture; and Others.General information on the application of the method and its results are showcased.