Augmented Reality in Ultrasound-Guided Regional Anaesthesia: An Exploratory Study on Models With Potential Implications for Training

Introduction Needle tip visualisation is a key skill required for the safe practice of ultrasound-guided regional anaesthesia (UGRA). This exploratory study assesses the utility of a novel augmented reality device, NeedleTrainer™, to differentiate between anaesthetists with varying levels of UGRA experience in a simulated environment. Methods Four groups of five participants were recruited (n = 20): novice, early career, experienced anaesthetists, and UGRA experts. Each participant performed three simulated UGRA blocks using NeedleTrainer™ on healthy volunteers (n = 60). The primary aim was to determine whether there was a difference in needle tip visibility, as calculated by the device, between groups of anaesthetists with differing levels of UGRA experience. Secondary aims included the assessment of simulated block conduct by an expert assessor and subjective participant self-assessment. Results The percentage of time the simulated needle tip was maintained in view was higher in the UGRA expert group (57.1%) versus the other three groups (novice 41.8%, early career 44.5%, and experienced anaesthetists 43.6%), but did not reach statistical significance (p = 0.05). An expert assessor was able to differentiate between participants of different UGRA experience when assessing needle tip visibility (novice 3.3 out of 10, early career 5.1, experienced anaesthetists 5.9, UGRA expert group 8.7; p < 0.01) and final needle tip placement (novice 4.2 out of 10, early career 5.6, experienced anaesthetists 6.8, UGRA expert group 8.9; p < 0.01). Subjective self-assessment by participants did not differentiate UGRA experience when assessing needle tip visibility (p = 0.07) or final needle tip placement (p = 0.07). Discussion An expert assessor was able to differentiate between participants with different levels of UGRA experience in this simulated environment. Objective NeedleTrainer™ and subjective participant assessments did not reach statistical significance. The findings are novel as simulated needling using live human subjects has not been assessed before, and no previous studies have attempted to objectively quantify needle tip visibility during simulated UGRA techniques. Future research should include larger sample sizes to further assess the potential use of such technology.


Introduction
Visualising the needle is a key skill for the safe practice of ultrasound-guided regional anaesthesia (UGRA), but teaching this whilst ensuring patient safety remains a challenge [1,2]. The operator must identify the needle tip before advancing and make any subsequent required adjustments as it approaches the target [3,4] Technology is emerging that may help track and visualise the needle tip, but it is not yet in widespread use [5,6]. There remains no agreed objective method to assess the standard of needle tip visibility during UGRA, and the authors are unaware of published data that formally quantifies operator expertise in this respect. It, therefore, remains difficult to objectively assess expertise in UGRA during training. Traditionally, technical (e.g., number of blocks preformed) and non-technical (e.g., management of the patient, recognising the limits of safe practice) factors have instead been used as surrogate markers [7,8].
A recent review article discussing future contemporary training methods in UGRA highlighted augmented reality as a key future avenue, and the Royal College of Anaesthetists UK has recently suggested the use of simulation and part-task trainers for learning in its most recent 2021 curriculum [9,10].
NeedleTrainer™ (Intelligent Ultrasound, Cardiff, UK) uses a blunt, retractable needle and augmented reality technology to superimpose a digital holographic needle on the real-time ultrasound image of a subject being scanned. This provides a simulated environment for the performance of UGRA needling skills on a real-life model. The software can also automatically calculate the amount of time the simulated needle tip is in view during any given procedure.
The device has not been validated to correlate with clinical competence or block success, so this exploratory study aims to provide data to inform future study design. However, objective knowledge of whether the needle tip is in view (using tracking technology) has been demonstrated to improve the block performance of novices [5]. Therefore, utilising simulation technologies to achieve this could be beneficial to larger groups training in regional anaesthesia.
The primary aim of this study was to determine whether there was a difference in needle tip visibility, as calculated by the device, between groups of anaesthetists with differing levels of UGRA experience.

Materials And Methods
This volunteer study was reviewed by the Office for Research Ethics Committees in Northern Ireland and deemed not to require NHS research ethics approval. The study received R&D approval via the Integrated Research Application System (IRAS; Project ID 299571).

Participants
Twenty anaesthetists from the Department of Anaesthesia, Royal Victoria Hospital, Belfast, provided written informed consent to participate in this study. Participants were stratified into four groups of five based on their experience in UGRA: novice (within the first 18 months of anaesthetic training), early career (from 18 months to 6 years into anaesthetic training), experienced anaesthetists (advanced trainees and non-expert consultants), and UGRA experts. In accordance with recent related studies, the definition of an expert was a consultant anaesthetist in the UK who met at least three of the following criteria [11][12][13]: (i) completed fellowship training in UGRA; (ii) holds a qualification related to UGRA (e.g., EDRA, higher degree, or equivalent); (iii) regularly delivers direct clinical care using UGRA, including for 'awake' surgery; (iv) regularly teaches UGRA, including regularly performing/teaching advanced techniques [14].

Models
Two volunteers provided informed written consent to be models for ultrasound scanning. The only exclusion criterion was the pathology of the areas to be scanned. The study was run over two days, with ten participants recruited each day and a different model used each day.

Expert assessor
One expert assessor (LT), meeting the criteria defined above, participated in this study. They were not blinded to the study participants.

Equipment
Ultrasound scanning was performed using a SonoSite X-Porte ultrasound machine (Fujifilm SonoSite, Bothell, Washington, USA) and a linear probe (HFL50xp). The NeedleTrainer™ device (v1.0) was connected to the X-Porte ultrasound machine via its high-definition multimedia interface output. Participants were asked to refer to the NeedleTrainer™ device screen, which displayed the ultrasound image with a superimposed virtual needle.
NeedleTrainer™ uses electromagnetic tracking of sensors placed on the ultrasound probe, along with information from the retractable needle, to infer the relative position of the virtual needle shaft and tip, and then plots that information onscreen (Appendices, Supplementary Data A) ( Figure 1).

Simulated UGRA protocol
All participants viewed a pre-recorded video prior to participation describing the three blocks to be preformed, the relevant endpoint for each block procedure, and an introduction to NeedleTrainer™. On the day of the study, hands-on familiarisation with NeedleTrainer™ was permitted prior to the assessment. This was limited to 10 minutes, with troubleshooting provided by the principal investigator (SS). Participants were permitted to practice with the NeedleTrainer™ on two block sites of the volunteer subject (the adductor canal and the forearm-level median nerve), neither of which were included in the study.
Participants then used NeedleTrainer™ to perform three simulated peripheral nerve blocks: interscalenelevel brachial plexus, rectus sheath, and popliteal-level sciatic nerve [15]. These techniques involved a range of body regions (upper limb, trunk, and lower limb) and needle endpoints (perineural versus fascial plane). The target for the final simulated needle tip position was pre-defined for each block (in the earlier prerecorded video) and represented a currently accepted standard in the literature [16,17].
Participants were free to set the ultrasound depth and gain settings deemed appropriate for each scan and use the scanning technique they would utilise in their normal clinical practice. A laminated sheet of the ideal sonographic image for all three techniques and defined NeedleTrainer™ simulated needle tip endpoints was provided to all participants. These images (Appendices, Supplementary Data B) were taken from a recent set of recommendations on structure visualisation [17], with an 'X' added to represent the intended final needle tip position. Assistance was permitted for initial image optimisation if required; however, there was no input once subsequent assessment with the simulated needle commenced.
The participant was timed for each procedure, from the point the retractile needle made contact with the subject's skin to the point at which the participant declared the final needle tip position had been achieved. This timing was recorded using a start/stop function on NeedleTrainer™, which allowed the device to calculate the needle tip visibility during this period. At the end of each simulated block, the expert assessor and participant rated the attempt using the outcomes listed below. This process was repeated for all three simulated blocks.

Primary Outcome
Objective NeedleTrainer™ data: Percentage time the simulated needle tip in view, calculated by the NeedleTrainer™ software (Appendices, Supplementary Data A).

Secondary Outcomes
Expert assessor data: (i) Time taken in seconds for each simulated block attempt (recorded as per the description above). (ii) Expert assessor judgement of simulated needle tip visibility (recorded after each attempt using a 0-10 continuous visual analogue scale: 0, never visible; 10, always visible). (iii) Expert assessor judgement of accuracy of the final simulated needle tip position (0-10: 0, very poor; 10, very accurate).

Sample size, data handling and analysis
As no previous studies have quantifiably assessed needle tip visibility during simulated UGRA, the authors had little data on which to base a sample size estimation. Studies using the Imperial College Surgical Assessment Device model for quantitative assessment of regional anaesthesia recruited 20-30 patients in total [19]. Therefore, a minimum of 20 participants was targeted.
Data are reported descriptively, and, where appropriate, statistical evaluation (Microsoft Excel version 2207, Build 15427.20210) was used to assess the relationship between variables. A two-factor ANOVA analysis with replication was used to compare the quantitative variables, apart from the assessment of the NASA task load index, where a single-factor ANOVA analysis was used. The Scheffe multiple comparison method was used to compare groups against each other. The null hypothesis stated that the level of experience does not influence any of the outcomes measured, with statistical significance defined as p < 0.05 (p-values are reported to 2 decimal places).

Results
Five participants were recruited for each group (four groups, n = 20), performing a total of 60 simulated blocks. Information on participants' experiences in UGRA can be found in Table 1 and the Appendices (Supplementary Data C).

TABLE 1: Participants self-rated experience in ultrasound-guided regional anaesthesia
Each participant was asked to fill in a pre-study questionnaire regarding their experience in UGRA. They were asked to self-rate their exposure to UGRA and their needling skills on a continuous scale from 0 to 10.
Both scan subjects were adult males; with body mass indexes (BMI) of 32 and 26 kg/m 2 . Primary and secondary outcomes are summarised in Table 2

Primary Outcome
Objective NeedleTrainer™ data: Level of experience was not associated with a statistically significant difference in the percentage of time the simulated needle tip was in view (p = 0.05; Figure 2). Expert assessment: Level of experience was associated with a statistically significant difference in time taken to perform simulated blocks (p = 0.02; Figure 3), the expert assessor's assessment of needle tip visibility (p < 0.01), and final needle tip placement (p-value < 0.01; Figure 4). Scheffe's analysis of the data found the UGRA expert group to be distinct from the other three groups in all three outcomes.   No variables assessed by the NASA task load index (mental demand, physical demand, temporal demand, performance, effort, and frustration) reached statistical significance when comparing stages of training (Supplementary Data D).

Discussion
The percentage of time the needle tip was in view, as per the NeedleTrainer™ software, was higher in the expert group (95% CI 50.4-63.7%) but did not reach statistical significance (p = 0.05). The highest percentage of needle tip visibility achieved across all 60 simulated procedures, 75%, was achieved by a participant in a non-expert group, with the mean in the UGRA expert group being only 57.1%. This figure is notable as it is lower than that predicted by the authors (no previous studies have assessed this), as most UGRA experts would expect to keep their needle tip in view throughout the majority of their initial needle advancement to a target endpoint. This may represent subtleties of clinical practice not captured in this study or represent a limitation of this simulated UGRA tool, and further studies with a larger sample size are required to determine whether this is a consistent or isolated observation.
The difference in the time taken to complete simulated blocks was statistically significant, with reduced time correlating with increased experience, in keeping with the authors' experience of clinical practice. The results support the assertion that an expert assessor is able to differentiate between different levels of operator experience, both in terms of needle visibility and simulated needle placement accuracy. The UGRA expert group took the least time and received the highest expert assessor scores. These data validate the method of human expert assessment and suggest that technology and human experts can be used in concert to assess operator performance in a simulated environment.
In comparison, the participants' subjective assessments were not able to differentiate between experience for both needle tip visibility (p = 0.07) and final needle tip position (p = 0.07). Whilst these data may suggest the concept that anaesthetists cannot accurately determine their own level of performance, validating the current practice of external assessment, this study was not powered to examine this endpoint.
To perform UGRA independently, the user must master real-life anatomy, sono-anatomy, image acquisition/interpretation, and needle tip identification [20]. Whilst the former can be learned safely through educational events and courses, the acquisition of needle-probe orientation skills can present a significant challenge for safe development. Indeed, it has been shown in novices that at least 28 supervised attempts at the same UGRA technique are required in cadavers to achieve competence in needle visualisation [21]. Anaesthetists must develop these skills for the care of future patients whilst minimising the risk to current patients. Many low-fidelity [22] and high-fidelity [23] simulated training devices have been developed for this purpose, including hybrid and cadaveric models, with studies citing flattened learning curves, a reduction in technical errors, and fewer needle passes following their use. As well as a lack of objective assessment data, criticisms include a lack of availability and cost, particularly for high-fidelity models, which to date are not in widespread use. NeedleTrainer™ is a readily transportable device, compatible with cart-based (v1.0) and handheld (v2.0) ultrasound machines, which expands the reach of this high-fidelity augmented reality simulator to users in their own institution. It allows repeated practice of simulated UGRA on real-life ultrasound scans/subjects and can be used immediately prior to clinical practice (e.g., to remind oneself of a particular procedure or to demonstrate/practice a safe intended needle path).
The potential to collect unbiased, objective data provided by such machines may also lend itself to formal assessment if validated in future studies. This objective metric may be our current primary outcome, percentage needle tip visibility during an entire simulated block, or another metric not assessed in this study, such as the dynamic percentage needle tip in view (i.e., only whilst the simulated needle is moving).
The authors recognise limitations in this work. The small sample size (60 simulated blocks) increases the risk of a type II error, and the investigation should therefore be repeated on a larger study population. Whilst the primary outcome was objective and free from bias, the secondary outcomes were subjective. In particular, the expert assessor was unblinded when making their determinations and so may have been prone to bias if they were aware of the participant's prior experience in UGRA. One way to prevent this in future studies is to use video recordings of each attempt, which can then be assessed by blinded expert assessors.
Participants were assessed following limited use of the device, and the study included only two scan subjects, assessing only three blocks. Therefore, future studies should include a greater range of procedures and a larger number of subjects with varying anatomical/physiological characteristics to reflect the range of clinical practice in UGRA and ensure the generalisability of these results. We also do not know if the percentage of simulated needle tip visibility necessarily correlates with clinical effectiveness, and future studies should be designed to evaluate this key outcome.
A significant challenge when designing the study protocol was categorising the four groups to be assessed, with grade of seniority used. Whilst this does not always correlate with experience/expertise in UGRA, we collected previous UGRA block numbers (Appendices, Supplementary Data C) and self-rating scores for each participant in a pre-study questionnaire. This demonstrated that in our dataset, the four groups correlated with increasing UGRA experience/exposure. Until UGRA exposure and training are standardised, it will continue to prove difficult to categorise expertise based on time in training or seniority of grade. Finally, the lack of haptic feedback through the retractile needle was noted as a potential limitation of the device compared to clinical practice, something offered by other high-fidelity simulators.

Conclusions
In this exploratory study, we evaluated the potential utility of a novel augmented reality device, NeedleTrainer™ to differentiate between anaesthetists with varying levels of UGRA experience on real-life models. The findings are novel as simulated needling using live human subjects has not been assessed before, and no previous studies have attempted to objectively quantify needle tip visibility during simulated UGRA techniques.
The primary outcome did not reach statistical significance (p=0.05) but may have been underpowered. In this study, the combination of the NeedleTrainer™ device and an expert assessor was able to differentiate between levels of experience in UGRA. This exploratory study provides further hypothesis-generating evidence, and future studies should be focused on the validation of an objective, non-bias metric of needle visualisation that could differentiate between UGRA experiences and therefore potentially assist standardised training in the future.

Appendices Supplementary data A
Further information on the NeedleTrainer™ System can be found below.
NeedleTrainer™ uses the Patriot electromagnetic tracking system from Polhemus [24]. This uses short-range electromagnetic fields to provide real-time tracking of the position and orientation of the sensor relative to a source transmitter (6-DOF tracking). In the NeedleTrainer™ setup, the sensor is attached to the probe, and the source is attached to the retractable needle, allowing the system to determine the relative position and orientation of the probe with respect to the retractable needle.
The ultrasound fan for the linear probe in NeedleTrainer™ is modelled as a thin virtual rectangular box with dimensions (probe width, probe depth, and ultrasound beam thickness). The NeedleTrainer™ system uses data from the tracker to calculate where the virtual needle is in relation to this box. The 'needle tip in view' metric is the proportion of time that the needle tip is inside the box, where the 'needle tip' is defined as being the last 2 mm of the needle.
The visibility of the virtual needle in this augmented reality setup is dependent on many different factors, such as the steepness of the approach (pitch axis -horizontal needles are easier to see), whether the needle is in-plane or out-of-plane (azimuth axis), the thickness of the virtual needle (low gauge needles are easier to see), the depth setting of the ultrasound machine, and the thickness of the box representing the fan (adjustable via NeedleTrainer's realism setting).

Supplementary data B
Below are pictures of the three laminated sheets provided to the participants during the online presentation and during the study session, with the target for the needle tip marked by an X (Figures 6-8).

FIGURE 6: Rectus sheath block
Laminated sheet provided to each participant during the online presentation and during the study session, with the target for the needle tip marked by an X.

FIGURE 8: Popliteal sciatic nerve block
Laminated sheet provided to each participant during the online presentation and during the study session, with the target for the needle tip marked by an X.

Supplementary data C
Each participant was asked to fill out a pre-study questionnaire regarding their experience at UGRA. This included the previous overall number of blocks preformed, the previous number of each block technique to be assessed, and a self-rating of UGRA exposure/needling skills on a continuous scale of 0-10. Self-rating data are found in Table 1 in the main body.

TABLE 3: Participant's previous ultrasound-guided regional anaesthesia numbers
Each participant was asked to fill in a pre-study questionnaire regarding their experience in UGRA. They were asked to estimate the number of UGRA blocks preformed to date using a modified Likert scale.