The Complexity of Online Patient Education Materials About Interventional Neuroradiology Procedures Published by Major Academic Institutions

Introduction Health literacy is an independent predictor of population health status and is directly related to the readability of available patient education material. The National Institutes of Health (NIH) and the American Medical Association have recommended that patient education materials (PEMs) be written between a fourth- and a sixth-grade education level. The authors assessed the readability of online PEMs about neurointerventional procedures that have been published by several academic institutions across the US. Methods Online PEMs regarding five common neurointerventional procedures, including mechanical thrombectomy for large vessel occlusion, cerebral diagnostic angiography, carotid artery stenting, endovascular aneurysm embolization, and epidural steroid injection collected from the websites of 20 top institutions in Neurology and Neurosurgery. The materials were assessed via five readability scales and then were statistically analyzed and compared to non-institutional education websites (Wikipedia.com and WebMD.com). Results None of the PEMs were written at or below the NIH's recommended 6th-grade reading level. The average educational level required to comprehend the texts across all institutions, as assessed by the readability scales, was 10-11th grade level. Some materials required a college-level education or higher. Material from non-institutional websites had significantly lower readability scores compared to the 20 institutions. Conclusions Current PEMs related to neurointerventional procedures are not written at or below the NIH's recommended fourth- to sixth-grade education level. Given the complexity of those procedures, significant attention should be pointed toward an improvement in the available online materials.


Introduction
Approximately 80 million Americans have limited health literacy, which puts them at greater risk for poorer access to care and poorer health outcomes [1]. Literacy, which reflects the ability to read and comprehend information, is directly related to readability, which is the ease with which a text can be read and understood [2]. Within the US general population, the average reading level has been found to be that of a seventh-or eighth-grade student, prompting the National Institutes of Health (NIH) and the American Medical Association (AMA) to recommend that patient education materials (PEMs) be written at a reading level no higher than the sixth grade [3]. In this study, we aim to evaluate the readability of online PEMs related to common neurointerventional procedures.
This article was previously presented as a meeting abstract at the 2022 SNIS 19th Annual Meeting in Toronto. Five common neurointerventional procedures were selected, including mechanical thrombectomy for large vessel occlusion, cerebral diagnostic angiography, carotid artery stenting, endovascular aneurysm embolization, and epidural steroid injection. The top 20 universities in Neurosurgery and Neurology based on the 2021 US News ranking were identified, and their websites were searched for online patient education material related to those procedures. Similarly, Wikipedia.com and WebMD.com, being common sources of non-institutional online patient education, were searched for similar materials for comparison purposes ( Table 1).  The text was reformatted using word-processing software (Microsoft Word, Microsoft Corp.) to exclude any navigational cues, hyperlinks, references, and images. An online readability analysis through WebFX Readability (https://www.webfx.com/tools/read-able) was used to assess the readability of the educational materials, using a series of five established scales: Flesch Reading Ease, Gunning Fog, Flesch-Kincaid, Coleman-Liau, Simple Measure of Gobbledygook (SMOG), and Automated Readability Index ( Table 2). SPSS version 26.0 (IBM Corp., Armonk, NY) was used to perform statistical analyses. The Kruskal-Wallis test was used to compare the average readability scores across the various scales and websites. Statistical significance was defined by a p-value < 0.05.

Results
The 20 included universities had a total of 76 online PEMs related to the studied procedures, including thrombectomy (n = 9), angiography (n = 17), carotid stent (n = 16), aneurysm embolization (n = 15), and epidural steroid injection (n = 19). Two additional materials on each procedure were obtained from Wikipedia.com and WebMD.com.
For each procedure, the readability scores from different tests compared to the recommended level by AMA and NIH are illustrated in Figure 1. The means and standard deviation for various readability scores of each procedure are reported in Table 3.

Discussion
The evolution of the internet has made it a major source of self-education for the public and allowed easier access to a wealth of online information. Currently, patients are increasingly using online PEMs for health information, with approximately 35% of American adults searching for diagnoses online. Readily accessible PEMs have been shown to increase healthcare utilization, preventative care, and patient-physician communication by providing a better understanding of diagnoses, symptomatology, treatment options, rehabilitation, and recovery, leading to improved health literacy [4][5][6].
Literacy has been touted as the single best predictor of healthcare status [5]. Poor health literacy is estimated to contribute to more than 73 billion dollars of additional burden to the US healthcare system [1,7]. The healthcare cost of Medicaid patients with limited literacy is about four times that of those who have adequate health literacy [8], as poor health-literate patients are assumed to be at high risk of exacerbation of their health problems, which in turn contributes to rising healthcare costs [9]. This burden, as well as the communication gap between patients and doctors, increases with the increased complexity of procedures, including neurointerventional procedures.
In this study, the mean readability scores for material published in major universities for the most common neurointerventional procedures were at 10-11th-grade level, which are 4-5 grades higher than the recommended level by AMA and NIH. Some of those materials were only comprehendible for the college level of education. When those materials were compared to similar materials obtained from noninstitutional sources like Wikipedia.com and WebMD.com, the latter two had significantly lower readability scores, making them more comprehensible to the public. However, in a study by Dutta-Bergman et al. which evaluated the trustworthiness of different sources of healthcare information on the internet, nearly half of the respondents revealed that they considered medical universities as one of the most trusted sources of online PEMs [10]. Yet, there remains a considerable shortcoming in those available materials. Furthermore, even websites that provide high-quality, evidence-based information, need to be written at a level appropriate for patients without extensive medical knowledge. In addition to quality, the readability of PEMs is an important consideration to ensure patients can understand and apply information related to their conditions and treatments.
Besides improving the general health of the public and the understanding of pathology and treatment, improving the readability of PEMs may also indirectly serve an important role in decreasing the frequency with which malpractice litigation occurs [2]. Improving the readability of PEMs could be achieved by minimizing the use of complex words, decreasing the number of words per sentence and syllables per word, using numbering or bullet points, and writing in an active voice [11]. Subsequently, this will lead to a better understanding of the procedure context and outcomes by the patients which will then improve the relationship between them and their interventionalists.

Limitations
This study is limited by the small number of institutions and materials included. It also does not consider other factors that can contribute to online material comprehension, including socioeconomic status, race, education, and income which can all contribute.

Conclusions
Online PEMs help improve healthcare utilization, preventative care, and patient-physician communication.
However, current materials related to neurointerventional procedures are not written at or below the NIH's recommended fourth-to sixth-grade education level. This would impact the utility of those PEMs for the general public. Given the complexity of those procedures, significant attention should be pointed towards an improvement in the available online materials by minimizing the use of complex words, decreasing the number of words per sentence and syllables per word, using numbering or bullet points, and writing in an active voice.

Additional Information Disclosures
Human subjects: All authors have confirmed that this study did not involve human participants or tissue. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue.

Conflicts of interest:
In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.