The 7 Elements Communication Rating Form for Teaching and Assessing Medical Communication Competency

Introduction: Medical communication skills are a critical component of clinical medicine and patient satisfaction. Communication skills are difficult to teach and evaluate, necessitating tools that are effective and efficient. This study presents and validates the 7 Elements Communication Rating Form (7E-CRF), a streamlined, dual-purpose, evidence-based medical communication checklist that functions as a teaching and assessment tool. Method: A 14-item teaching and assessment tool is described and validated using face, concurrent, and predictive validity indices. The study was conducted with 661 medical students from the West Virginia School of Osteopathic Medicine (WVSOM). Student performance was assessed in year 1 labs, year 2 labs, and year 2 and year 3 objective structured clinical examination (OSCE). These internal indices were compared with student performance on the Humanistic Domain of the Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2-Performance Evaluation (PE), a licensure exam previously taken in years 3 or 4 of osteopathic medical schools. Results: The evidence of interrater reliability and predictive validity is strong. Data from the 7E-CRF is compared to performance on the COMLEX Level 2-PE, Humanistic Domain. The 7E-CRF can identify students who are at a 10-fold increased risk of failure on the COMLEX Level 2-PE Humanistic Domain. Conclusions: The 7E-CRF integrates instruction and assessment, based on a national and international model. The simplicity, foundation in professional consensus, ease of use, and predictive efficacy make the 7E-CRF a highly valuable instrument for medical schools in teaching and evaluating competency in medical communication skills.


Introduction
There has been increased attention to medical communication skills in medical education stemming from the core competency movement promulgated by the Accreditation Council of Graduate Medical Education [1].Gaining momentum since the turn of the millennium, this important area has been in need of tools for assessing medical communication skills as well as a concise training model for novice learners in medicine and advanced practitioners alike [2].Over 70 published tools exist to assess medical communication skills with varying theoretical bases, design strategies, and reliability data [3,4].
Bedside manner has been a construct in the discussion of doctoring throughout the 20th century and into the new millennium [5,6].Many people have family members who have been treated by doctors who they regard as having wonderful bedside manners and others they find lacking in this critical domain.Indeed, patient satisfaction surveys report that the doctor's bedside manner is a critical factor in choosing a practitioner [7].Prior to 2000, there was substantial heterogeneity in training on healthcare communication strategies with different schools teaching different concepts, and with different professors within a particular medical school teaching different approaches.Such variety and variability did not lend itself to standardization and competency-based approaches that were growing in medical education [8].
The culture of academic medicine has increasingly elevated doctor communication skills as part of its purview [9].Standardized curricula for teaching medical communication skills and tools for measuring such skills are essential, particularly as communication competency and the evaluation of communication skills has been a requirement for medical licensure in both the Osteopathic and Allopathic professions [10].Although both Doctor of Medicine (MD) and Doctor of Osteopathic Medicine (DO) licensure organizations have pulled back from clinical examinations (partly due to COVID-19 pandemic considerations), both brought about a desperately needed professional agreement that detailed the "essential elements of healthcare communication" [11].These seven elements are 1) open the discussion, 2) build the relationship, 3) gather information, 4) understand the patient's perspective, 5) share information, 6) reach agreement on problems and plans, and 7) provide closure.These elements also form the backbone of the didactic material on DocCom, a widely used internet platform for teaching medical communication skills produced by the Academy on Communication in Healthcare [11,12].
The objective of this study is to describe and validate an effective rating score for accessing clinical communication skills of osteopathic medical students and residents.An instrument was designed with the goals of reliability, validity, brevity, clarity, and facility of training and use.

Setting
The

Design of the 7E-CRF
In light of the elegant framework provided by both the Kalamazoo Consensus Statement and DocCom, WVSOM chose the structure of the essential elements for its didactic and evaluative communication tool development [11,12].The seven essential elements formed the structure of the assessment tool and training model.On the 7E-CRF, each essential element is a domain with two specifically designed items creating a 14-item scale.For the design of the items, the committee had three considerations (1) face validity and consistency of the item with the description provided in the Kalamazoo Consensus statement, (2) research evidence of the item, and (3) ease of training for both raters (standardized patient (SP), faculty, students and administrators) and learners.
The inclusion of two scored items for each of the elements resulted in an evaluation tool with 14 items for grading in seven domains (Table 7).Among the 14 items, some are more "objective" while others are more "subjective."The objective in this sense means that they are more clearly anchored in discrete, observable behaviors (e.g., did the student introduce themselves by name).The more subjective items, on the other hand, are multi-faceted, nuanced, and essentially a Likert Scale with the categories of poor (1), fair (2), good , and excellent (4).These more subjective items tend to be more global in nature, have a higher level of consideration grading, and require the SP to make a judgment based on their experience and expertise in grading tens to hundreds of students, thus comparing a new student to an internalized sense of the distribution of performance.The SPs receive orientation training on the form, written grading guidelines, and continued monthly training on the communication rating form in addition to the specific case training, following Aspe's best standards of practice [13].
For each of the 14 items on the scale, the grader must select a response of 1, 2, 3, or 4.These are not all associated with consistent qualitative words since, as described above, some of the items fan out across a qualitative descriptive gradation, while others relate to the presence or absence of identified target behaviors in the sample of student behavior.These point values are then recalibrated to a 10-point scale for evaluative and grading purposes with 1 = 4, 2 = 6, 3 = 8, and 4 = 10.With this calibration, we are able to render scores on a 10 or 100-point distribution which works well with WVSOM grading practices with a score of 70% to 75% generally used for passage of various high-stakes assessments.The 14-item 7E-CRF translates the scores to a scaled score with 10 as perfect performance per item which provides for a maximum score of 140 on the overall instrument.This is converted to a 100-point or percentage score which corresponds well to assessment in the WVSOM curriculum.

Use of the 7E-CRF for teaching and evaluation
Students receive instruction on the 7E-CRF beginning the first week of their first year in medical school and have continuous access to the form.The decision was made to disregard the students' educational level when assigning scores on the 7E-CRF and use the 7E-CRF as a consistent measurement tool.The rationale for this decision is two-fold.First, the reliability of ratings on a measure of such complex, interactive human behavior is always a challenge and requires intensive, continuous training.Asking raters (primarily SPs) to alter their scoring judgments based on a student's year in the curriculum would add an unnecessary challenge and unreliability to the scoring activity.This is compounded with the fact that the 7E-CRF is also used for residents and practitioners; so differentially adapting a unique version of the scoring judgments for each group seemed a cumbersome invitation to "police" and reduced reliability.Also, the concept of a consistent measurement tool allows student progress over the course of the curriculum to be followed.In this model, a student, and the institution, can see how a student's skills change over the course of their education and clinical exposure, and also in relation to experimental educational interventions and curricular changes.
The intensive training and reliability process for the SPs has been largely conducted as a live, group activity with a detailed manual for guidance.The manual has scoring guidelines and examples for each of the 14 items as well as philosophical orientation and justifications of the approach.The manual provides the SPs with insight into how the students are trained in communication.Knowing the information the student is given beforehand reassures the SP that the student is being graded fairly and encourages them to grade accurately.The information in the manual is crucial for the consistency of graders [14].
The reliability of the 7E-CRF is of ongoing importance.Continual training is essential to maintain the interrater reliability of a group of 60 to 80 SPs.A "Reliability Workshop" is held monthly where SPs come together for three hours of training with dedicated faculty and simulation center staff.The Reliability Workshop is a costly endeavor but necessary to maintain interrater reliability.
During the workshop each month, one case from our year 3 OSCE is reviewed.Three videos of this case are observed and all the present SPs grade the student using the 7E-CRF.After grading is complete, each item is verbally reviewed and the SPs are able to visually see if they are with the majority or could possibly be grading too hard or too easy.The discussion gives the SPs the ability to understand what is important for each question and how others may have interpreted the same actions.The data gathered at each Reliability Workshop is collected and used to evaluate the SPs individually and as a group.

7E-CRF performance
Overall average scores on the 7E-CRF have been analyzed by curricular year for the class of 2016, 2017, and 2018.Table 1 shows the average 7E-CRF scores for an eight-station OSCE for year 2 and a 12-station OSCE for year 3.  On average, students are scoring between 7.8 and 8.0 on each item, which corresponds to the Likert designation of just under a "3."For the class of 2016 and 2017, there appears to be a slight trend toward improved 7E-CRF performance from year 2 to year 3 OSCE, but this was not found in the class of 2018.No significant effect was found for the gender of students with male and female averages for the class of 2017 during the year 3 OSCE being very similar (female = 82.6;male = 82.0).

7E-CRF predictability of pass vs fail on Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2-Performance Evaluation (PE)
The This similar analysis is presented in Tables 2, 3     The 7E-CRF status of high versus low performance was predictive of risk for COMLEX 2-PE failure with lower performance being 7 to 25 times more likely to fail as compared to 7E-CRF high performers.

7E-CRF reliability analysis
Data from SP Reliability Workshop monthly coding of year 3 OSCE encounters in 2016 (class of 2017) indicate interrater reliability range from Cronbach's Alpha of 0.69 to 0.83 on individual third-year OSCE stations [15].As seen in Table 6, in 2017, year 3 OSCE stations' 7E-CRF reliability values were mostly above 0.70 value.Overall internal reliability for the 7E-CRF is calculated at a Cronbach's Alpha value of 0.75.

Discussion
In this study, a tool for teaching and assessing healthcare communication is described [16].The use of the same tool for both teaching and evaluation is a critical component of the 7E-CRF's success.This "dual-use" provides a continuous loop between instruction and evaluation that is well-suited for adult learners.The reliability of the 7E-CRF is good, and it is also valid with significant predictive validity for identifying at-risk status for the Humanistic Domain of the COMLEX Level 2-PE.
Comparing the 7E-CRF to other instruments, the fundamental advantages are its simplicity (both the teaching model and the evaluation are the same two-page document), the dual-use function (which closes the loop for adult learners), the existence of a detailed SP user manual and proven efficacy in identifying atrisk individuals.An additional aspect upon comparison is the osteopathic philosophy of treating the whole person, whereas each individual is a unified combination of mind, body, and spirit [17].

Conclusions
The 7E-CRF accomplishes the intended goals of transparency for adult learners, based on the internationally accepted Kalamazoo Model for best practice, ease of use, and strong reliability and validity.Thus, it is a prime tool for adoption across educational and healthcare settings.

Appendices 7 Elements: WVSOM Communication Rating Form
Under each item number, there are descriptions of each score level, 1 being the lowest and 4 being the highest.These are given as a general anchor point.Many times, the situation will not fit perfectly into one of these 4 ratings.Also, the grader has the flexibility to raise or lower a grade around this "target example".For instance, if the student meets the stated criteria for a score of 3 on Item #9 (Gives and explains a diagnosis/condition/symptoms/etc.), but uses a lot of jargon, then the grader could bump that grade down from a 3 to a 2.) Open the Discussion 1 Greet Patient (greet patient by name, introduce self, eye contact/smile, and sit down) these 4 things must be done prior to eliciting the chief complaint.
1-Performs 2 or fewer prior to eliciting chief complaint.
NBOME and NBME) continue to work with medical schools to standardize and ensure communication competency as part of their charge to protect the public.The increased importance of communication competency and high-stakes examinations made standardization across medical schools critical.In 2001, The Kalamazoo Consensus Statement, by the participants in the Bayer-Fetzer Conference on Physician-Patient Communication in Medical Education, predictive validity of the 7E-CRF was analyzed by comparing WVSOM year 3 OSCE scores on the 7E-CRF to student performance on the National Board of Osteopathic Medical Examiners COMLEX Level 2-PE, Humanistic Domain.WVSOM has consistently performed well on the COMLEX Level 2-PE Humanistic Domain with pass rates in the 97-98% range (class of 2016 = 97.5%;class of 2017 = 97%; class of 2018 = 98%).With such a large percentage of the student body passing this critical exam, we have looked closely at those who do not pass and considered their 7E-CRF scores to determine if predictions could be made.Among the class of 2017 (sample size 177), there was a statistically significant (p < 0.001) difference between 7E-CRF scores for the year 3 OSCE students who passed the Humanistic Domain (82.6) and those who failed the Humanistic Domain (75.9) of the COMLEX Level 2-PE.A useful method for analyzing the data is to convert 7E-CRF scores via a cutoff score of 75 into categorical variables such as high and low performers.Using this dichotomy and comparing the high and low performers with COMLEX 2-PE pass/fail yields a 2 × 2 matrix of hits and misses.In the class of 2018 year 3 OSCE, 15 out of 198 students scored <75 on the 7E-CRF for year 3 OSCE.Two of the 15 failed the PE Humanistic Domain and 13 did not.Additionally, two students who scored above 75 on the 7E-CRF failed the PE Humanistic Domain.This data can be analyzed in terms of "hits" and "misses."(a) Low performers on the 7E-CRF had a 2 in 15 or 13% chance of failing the Humanistic.(b) High performers on the 7E-CRF had a 2 in 183 or 1.1% chance of failing the Humanistic.(c) Therefore, low performance on the 7E-CRF is associated with an approximately 11.8-fold increased risk of failing the PE Humanistic Domain.

3 - 4 - 1 -
Performs 4/4 required elements prior to eliciting chief complaint.Performs 4/4 required elements prior to eliciting chief complaint and shows very high sincerity and engagement.2Elicitchief complaint and any other concerns using open-ended questions and empathy."What brings you in today?" " I'm sorry to hear that.""Can you tell me more about that?" "Is there anything else you would like to discuss today?"Begins with 1 open-ended question.2-Beginswith 2 open-ended questions.3-Asks 3 open-ended questions before the physical exam.4-Asks 3 open ended-questions before physical exam and makes an empathy statement.2024 Wise et al.Cureus 16(4): e59008.DOI 10.7759/cureus.590086 of 9 study was conducted with 661 medical students from the West Virginia School of Osteopathic Medicine (WVSOM), Lewisburg, United States in the classes of 2016, 2017, and 2018.

TABLE 1 : Average 7E-CRF score by graduating class for year 2 and year 3 OSCE The
average scores for a second-year eight-station OSCE and a third-year 12-station OSCE for academic years 2016, 2017, and 2018.OSCE: objective structured clinical examination; 7E-CRF: 7 Elements Communication Rating Form , 4 for each of the graduating classes of 2016, 2017, and 2018.

TABLE 4 : Humanistic Domain result by 7E-CRF status-class of 2018 (n = 198)
7E-CRF: 7 Elements Communication Rating Form; PE: Performance Evaluation The increased risk of failing the COMLEX Level 2-PE Humanistic Domain is substantial for individuals who 2024 Wise et al.Cureus 16(4): e59008.DOI 10.7759/cureus.59008score at or below an overall year 3 OSCE 7E-CRF score of 75%.The increased risk for each of these classes is shown in

TABLE 5 : Increased risk of failing the COMLEX Level 2-PE Humanistic Domain as a function of a low 7E-CRF score
The increased risk of failing the humanistic domain is substantially increased for students scoring at or below 75% as shown in the table.

TABLE 6 : Interrater reliability by OSCE station
Year 3 OSCE stations for the class of 2017 indicate an interrater reliability range of Cronbach's Alpha of 0.69 to 0.83 with an overall value of 0.75.
To fully engage at the "whole person" level, practitioners need effective communication instruction, feedback, and evaluation; the 7E-CRF provides this.Students who score at or below an overall score of 75 on the 7E-CRF are required to attend additional training in medical communication skills prior to taking the COMLEX Level 2-PE.This re-education program provides additional training and practice with the 7E-CRF model.Without this additional training, many more students with low 7E-CRF scores may have had difficulty at the COMLEX Level 2-PE.It is our hypothesis that this intervention program for these at-risk students contributed to our "false positive" numbers in the analysis in Table4.Furthermore, in this application, false positives carry little negative consequence viz two hours of extra communication training; for this reason, minimizing false positives (at the expense of fewer hits) is not recommended.One of the goals of the 7E-CRF is that the tool stands as a highly condensed pedagogical summary of a nationally acknowledged communication model.It is presented repeatedly in student training in a simplified, streamlined, and repetitive spiraling curriculum allowing students to clearly understand and exhibit the communication behaviors they will be tested on.It is a teaching tool in addition to an assessment measure.The students' attention is frequently called to the fact that this content has been condensed down to a two-page format and that it has a very high yield in terms of reliably predicting board passage while also being brief and easy to use by multiple assessors (students, faculty, and SPs).The recent change in the elimination of the practical examination component of the COMLEX licensing exam will require medical schools to ensure student competency in clinical skills.The 7E-CRF is demonstrated to be a valid evaluation tool with established cut-off scores that correspond closely to the previous vigor of the COMLEX 2-PE Humanistic Domain standards.Continued studies will need to be completed to further revise the 7E-CRF to accommodate the telehealth platform and additional studies on the competency needed for those encounters.