Diagnostic Reliability of Leptomeningeal Disease Using Magnetic Resonance Imaging

Brain metastases are seen in 20%-50% of patients with metastatic solid tumors. On the other hand, leptomeningeal disease (LMD) occurs more rarely. The gold standard for the diagnosis of LMD is serial cerebrospinal fluid (CSF) analyses, although in daily practice, the diagnosis of LMD is often made by neuroimaging. Leptomeningeal metastases (LM) have been a relative contra-indication to radiosurgery. It can be noted that focal LMD can be difficult to distinguish from a superficially located/cortical-based brain metastasis which is not a contra-indication for radiosurgery. Hence, justifying the need of a reliable diagnosis method. The goal of this study was to determine the inter-observer reliability of contrast-enhanced magnetic resonance imaging (gdMRI) in the differentiation of focal cortical-based metastases from leptomeningeal spread. This is a retrospective review of a prospectively collected database of patients with brain metastases. A total of 42 cases with superficial lesions were selected for review. Additionally, eight control cases demonstrating deep and/or white-matter based lesions were included in the study. Three neuroradiologists and three radiation oncologists were asked to review each study and score the presence of LM. Inter-observer agreement was calculated using group-derived agreement coefficients (Gwet’s AC1 and Gwet's AC2). Pair-wise inter-observer agreement coefficients never reached substantial values for trichotomized outcomes (LMD, non-LMD or indeterminate) but did reach a substantial value in a minority of cases for dichotomised outcomes (LMD or non-LMD). The control subgroup analysis revealed substantial agreement between most pairs for both trichotomized and dichotomised outcomes. We observed low inter-observer agreement amongst specialists for the diagnosis of focal LMD by gdMRI. Neuroimaging should not be relied upon to make treatment decisions, notably to deny patients radiosurgery.

The goal of this study was to determine the inter-observer reliability of contrast-enhanced magnetic resonance imaging (gdMRI) in the differentiation of focal cortical-based metastases from leptomeningeal spread. This is a retrospective review of a prospectively collected database of patients with brain metastases. A total of 42 cases with superficial lesions were selected for review. Additionally, eight control cases demonstrating deep and/or white-matter based lesions were included in the study.
Three neuroradiologists and three radiation oncologists were asked to review each study and score the presence of LM. Inter-observer agreement was calculated using group-derived agreement coefficients (Gwet's AC1 and Gwet's AC2). Pair-wise inter-observer agreement coefficients never reached substantial values for trichotomized outcomes (LMD, non-LMD or indeterminate) but did reach a substantial value in a minority of cases for dichotomised outcomes (LMD or non-LMD). The control subgroup analysis revealed substantial agreement between most pairs for both trichotomized and dichotomised outcomes.
We observed low inter-observer agreement amongst specialists for the diagnosis of focal LMD by gdMRI. Neuroimaging should not be relied upon to make treatment decisions, notably to deny patients radiosurgery.

Introduction
Brain metastases occur in 20%-50% of patients with advanced cancer. Leptomeningeal disease (LMD) is diagnosed in 5%-10% of patients with otherwise metastatic cancer, although rates approaching 20% have been reported in autopsy series [1][2]. The gold standard for the diagnosis of LMD is serial cerebrospinal fluid (CSF) analyses, which has a sensitivity of approximately 55% on the first CSF examination and reaches up to 85% sensitivity when done thrice -while maintaining a theoretical specificity of 100% [3][4]. The presence of malignant cells in a sample of CSF, cranial nerve palsies or widespread meningeal enhancement (particularly in the cerebellar folia) is convincing evidence of a process that is not amenable to focal treatment. On the other hand, nodular lesions described as being of leptomeningeal origin can cause more therapeutic ambiguity. In current practice, diagnosis of LMD is also often made without CSF analysis, using gadolinium-enhanced magnetic resonance imaging (gdMRI) of the brain and/or spinal cord, which is reported to have a sensitivity and specificity of approximately 75% [5][6]. LMD thus diagnosed can represent a relative contra-indication to stereotactic radiosurgery.
The goal of this study is to characterize the inter-observer reliability of gdMRI to differentiate focal cortical-based metastases from focal LMD in the context of known metastatic disease.

Materials And Methods
This is a retrospective study of cerebral magnetic resonance imaging (MRI) studies of patients suspected to have focal leptomeningeal metastases (LM). All patients included in this study underwent radiosurgery between 2009 and 2013. The study protocol was approved by our institutional ethics review board.
We completed a retrospective review of a prospectively collected database of 438 patients with known primary disease and neurological symptoms referred for evaluation to our tertiary care center radiation-oncology clinic between May 2009 and June 2013. Of these patients, 174 had a reviewable planning MRI prior to intervention in radiation-oncology. These pre-intervention studies were screened randomly by a single author (FG, neuroradiologist with 15 years of experience) until 42 cases were selected for review, while blinded to the original imaging report. The sole inclusion criteria was the presence of one or more enhancing cortical-based lesions. Cases demonstrating widespread leptomeningeal or cranial nerve enhancement, as well as cases with previous radiosurgery or surgery were excluded from the study, as were cases with suboptimal study quality. Additionally, eight control cases were included in the study, consisting of eight patients demonstrating only deep and/or white-matter based lesions.
Readers included three neuroradiologists (two senior staff members (5-18 years of experience) and a neuro-radiology fellowship trainee) and three radiation oncologists with active radiosurgery practices (5-15 years of experience).
All MRIs studies were independently reviewed by each of our six examiners using our institution's Picture Archiving and Communication System (PACS), and were assigned to one of three diagnostic categories (LMD, non-LMD, indeterminate). To establish a diagnosis per specialty, cases who received the same diagnostic label by two readers were assigned to that category, while cases who received three different diagnoses were re-evaluated by the three specialists to establish a consensus diagnosis.
In addition to analysing the data using a ternary scale (LMD, non-LMD, indeterminate), statistical analyses were also performed using a binary scale, in which cases assigned to the "indeterminate" category were clustered with cases assigned to the "non-LMD" category, based on the assumption that patients without a firm diagnosis of LMD would be treated in a similar fashion in regards to radiotherapy.
Inter-rater coefficients were calculated between each pair of readers using Gwet's AC1 and AC2, for dichotomized and trichotomized ratings respectively, due to their more robust nature, allowing inter-observer analyses on data demonstrating a low prevalence index and presence of constants, notably in the controls. Gwet's AC1 and AC2 was also used to derive group interrater coefficients amongst the entire cohort, as well as subgroups (amongst radiologists, radiooncologists and between the two specialties, using the consensus agreements described above).

Results
While some cases achieved an agreement by all readers, some other cases presented more ambiguous lesions thus causing division amongst readers (Figures 1-2). In trichotomized ratings, only one out of the 42 cases and only three out of the eight controls were assigned to the same diagnostic category by all six raters. All other studies were assigned to two different diagnostic categories (out of three) by at least two raters, with variable agreement amongst the rest of the raters. In dichotomized ratings, these numbers rose to nine out of 42 cases and four out of eight controls. In both trichotomized and dichotomized ratings, only the non-leptomeningeal category was assigned by all six raters. No consensus was obtained within the leptomeningeal category.

Discussion
Cortical-based metastases are difficult to differentiate radiologically from focal leptomeningeal carcinomatosis. Our study demonstrates significant variability in the interpretation of studies demonstrating cortical-based enhancing disease, both amongst clinicians and radiologists.
However, inter-rater agreement was nonetheless higher amongst neuro-radiologists (moderate) than radio-oncologists (slight), suggesting increased reliability of certain radiological signs, best evaluated with radiological expertise.
LMD remains a clinical diagnosis, based on clinical symptoms, imaging, and cerebrospinal fluid analysis. Presence of CSF cytology remains the gold standard, with 60% sensitivity on initial puncture, 80% total sensitivity on second puncture and 2%-5% sensitivity increase per following repeated collection [9]. Most diagnostic algorithms recommend initial evaluation with a CSF profile, CSF cytology and gdMRI of the neuroaxis, with at least one additional lumbar puncture if initial CSF cytology is negative [9][10][11][12]. Meanwhile, the frequency of LMrelated MRI abnormalities for solid tumors on neuroaxis imaging varies from 56% to 65%, depending on the primary [13]. Although in some cases, the constellation of clinical symptoms of LMD combined with bulky LMD on neuroaxis imaging may obviate the need for lumbar puncture, our study shows that, in more mitigated cases, focal cortical-based metastases are difficult to differentiate from focal LMD [3][4]6]. In such cases, we advocate that imaging findings should be correlated by CSF analysis, serial of need be, before LMD can be diagnosed, in large part because of the implication for treatment, most notably radiotherapy.
Although there is no universally accepted standard treatment for LMD, whole-brain radiotherapy (WBRT) is often considered in this patient population. While some cases can benefit from WBRT, this treatment does expose healthy brain tissue to high doses of radiations [14]. Adverse effects of cerebral radiation therapy may include neurocognitive disorders, radionecrosis and, rarely, radiation-associated meningiomas (RAM) [15]. Alternatively, stereotactic surgery (SRS) achieve high rate of local control while diminishing radiation exposure to the rest of the brain [2].
Ross et al. outlined the importance of a clearly established diagnostic before exposing patients to SRS or WBRT, as the implications of misdiagnosis might either be radiation overexposure or missed tumors [16]. Flickinger et al. argue that imaging alone represents an accepted diagnostic tool although they report a 2.3% chance of misdiagnosis, when imaging is not confirmed by CSF analysis [17]. Alternatively, Wolf et al. suggest that the use of SRS might successfully treat focal LMD and delay WBRT for some patients [2]. Consequently, radiosurgical approach has been used as a treatment for focal LMD [2,18].
Therefore, because LMD is often difficult to differentiate from cortical based metastases and because initial focal radiotherapy may delay whole-brain radiotherapy, we argue that patients with a known solid primary neoplasm and equivocal imaging findings for LMD should still be considered for stereotactic surgery initially.

Conclusions
The diagnosis of nodular LMD as opposed to cortical-based brain metastases on gadoliniumenhanced brain MRI demonstrates low inter-observer agreement amongst specialists, both radiologist and radio-oncologists. If imaging findings may alter the treatment approach, they should at least be confirmed by serial CSF analysis.

Additional Information Disclosures
Human subjects: Consent was obtained by all participants in this study. Comité d'éthique de la recherche en arts et en sciences du Centre Hospitalier de l'Université de Montréal (CÉRAS) issued approval 15.111. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.