#OrthoTwitter: Relationship Between Author Twitter Utilization and Academic Impact in Orthopaedic Surgery

Background #OrthoTwitter has evolved to disseminate findings and engage the public. However, the academic impact of Twitter utilization in orthopaedic surgery is unknown. Questions/purposes The purpose of the study was to evaluate relationships between the author and manuscript Twitter activity and citations. Methods Manuscripts in 17 orthopaedic journals from 2018 were identified. Citations, online mentions, impact factors, and subspecialties were obtained. H-index and Twitter account details for authors were obtained for a subset of manuscripts. Relationships between Twitter activity and citations were evaluated. Results 2,473/4,224 (58.5%) manuscripts were mentioned on Twitter (n=29,958 mentions), with Twitter manuscripts cited more frequently (median 10 vs. 7, p<0.0001). Twitter mentions, impact factors, non-open-access status, and subspecialties were associated with citation counts. Articles mentioned in 10, 100, and 1,000 Tweets were observed to have a 1.1-fold, 1.7-fold, and 245-fold increase in citations. In author-level analyses, 156 (20.0%) first and 216 (27.7%) senior authors had Twitter accounts. Citation count was associated with increasing senior author H-index (βest=0.13, p<0.05), Twitter mentions (βest=0.0043, p<0.0001), impact factors (βest=0.13, p<0.0001), and having a first (βest=0.20, p<0.05) or senior author (βest=0.17, p<0.05) on Twitter. Articles published in arthroplasty (βest=0.49, p<0.05), general interest (βest=0.55, p<0.01), sports (βest=0.63, p<0.01), and non-open access journals (βest=0.41, p<0.001) were cited more. H-index correlated with followers for first (rho=0.31, p<0.0001) and senior authors (rho=0.44, p<0.0001). Conclusion Author Twitter utilization is independently associated with manuscript citations. Authors should be aware of the potential association between social media utilization and traditional academic impact. Understanding the relationship between social media utilization and academic impact is necessary to effectively disseminate research.


Introduction
Social media is transforming the way academic medical research is shared. While articles are shared on a variety of platforms, more than four out of five online mentions of academic manuscripts occur on Twitter [1]. With over 400 million users and a brief character content sharing limit, Twitter offers a substantially greater audience and speed of dissemination compared to academic journals. Within the field of orthopaedic surgery, the "#OrthoTwitter" community on the Twitter platform has evolved as a forum to disseminate findings, exchange ideas, and connect with both the public and other practitioners [2][3][4]. It has been suggested that Twitter activity at the time of publication is associated with academic citation counts after several years [1,5,6]. However, the academic impact of social media utilization, particularly by authors, remains uncertain in the orthopaedic surgery community.
Orthopaedic surgery is also comprised of multiple diverse subspecialties -each with its own subset of journals and communities -which may influence the relationship between Twitter utilization and citations. While citation counts and online mentions have previously been reported for hand surgery [6] and sports medicine [5], cross-subspecialty comparative analyses are limited. It remains to be known how the relationship between online utilization and citations may vary among orthopaedic subspecialties.
Furthermore, while alternative metrics (e.g., Altmetric Attention Score [AAS]) have been proposed to modernize the estimated impact of manuscripts [7], there is no corresponding method of adjusting the impact of authors based on their individual online influence. Author-level productivity has historically been assessed using the Hirsch index, which is derived from the citation count of published manuscripts [8]. While authors maintain varying degrees of influence on social media, less is known about the relationship between online popularity and academic credibility and impact. For example, a recent analysis found 22 of the 100 most influential orthopaedic surgery Twitter accounts were not orthopaedic surgeons [9].
Understanding the association between social media utilization and academic impact and how this association may vary across subspecialties is increasingly necessary to effectively disseminate research and assess research impact. The purpose of the study was to evaluate the relationship between citation count and both manuscript-related Twitter activity and author Twitter utilization among orthopaedic surgery subspecialties. We hypothesized that author Twitter utilization and manuscript-related Twitter activity would be associated with greater citation counts and that there would be variability among orthopaedic subspecialties.

Data collection
Because this study utilized publicly available data, no Institutional Review Board approval was sought. No funding was received for this research.

Article Selection
Seventeen high-impact orthopaedic surgery journals, representing a range of subspecialties, were selected. Open access status, subspecialty, and impact factor were recorded ( Table 1). Articles published in these journals between January 1, 2018 and December 31, 2018 were identified using the easyPubMed R package (Vienna, Australia) to query PubMed (https://pubmed.ncbi.nlm.nih.gov/) [10]. The authors, titles, abstracts, and digital object identifiers (DOI) were extracted from the identified article metadata using the same package. Letters to the editor, editor's notes, responses, errata, and technique videos were excluded. Because the analysis was performed in January 2022, articles published in 2018 were selected to overcome citation lag [11,12]. Article citation counts were obtained using the rscopus R package (RStudio, Boston, MA) [13].   [14]. The rAltmetric R package was used to query the Altmetrics database for the AAS and online article mentions [15].

Calculation of Author Hirsch Index
Hirsch developed an author-level measure of research impact, termed the h-index, which, although biassed towards researchers with longer careers, is commonly reported in bibliometric analyses [8]. An author's hindex is defined as the integer, h, such that the author has at least h publications with at least h citations each. The h-index was calculated for the first author and senior author of each publication.

Author-Level Twitter Information
Twitter account handles were recorded for the first and senior authors for a random subset of 20% of the included manuscripts. Senior authors were assumed to be the last authors listed on a given manuscript. The subset was selected by sorting the articles in a randomized order and selecting the top 20%. The sample size was selected to balance feasibility with statistical power. The Twitter API (Twitter, San Francisco, CA) was employed to obtain author-level account details from the author account handles, including the number of account followers, the number of accounts followed, the number of tweets, and the account date of creation.

Statistical analyses
Data were analyzed using R version 4.1.2 (Vienna, Australia) [16]. Normally distributed data are summarized using the mean and standard deviation. Skewed data are summarized using the median and interquartile range (25th and 75th percentiles). To assess the correlation between continuous variables, Spearman correlation coefficients were calculated. To assess relationships between multiple groups (e.g., subspecialties), a Kruskall-Wallis one-way analysis of variance was performed. The correlation coefficients were interpreted according to Landis and Koch as none (<0), slight (0.01-0.2), fair (0.21-0.4), moderate (0.41-0.6), substantial (0.61-0.8), and almost perfect (0.81-1) [17]. P-values less than 0.05 were considered significant. All tests of significance were two-tailed.
The relationship between citations and manuscript-level data was assessed for the full set of manuscripts. The relationship between citations and manuscript-and author-level data were assessed for the specified randomized subset of 20% of manuscripts. Negative binomial logistic regression models were generated via generalized linear modelling to assess the relationship between citation count and author-and manuscriptlevel data. Negative binomial logistic regression models were selected as citation counts were overdispersed. P-values were generated using Wald tests. In this approach, the presented beta coefficients represent the expected change in the natural log of citation count for a given one-unit increase in the predictor. A forced entry approach to variable selection was employed because associations between citation count and h-index, open access status, journal impact factor, Twitter mentions, and subspecialty have previously been suggested or hypothesized [4,18,19]. For consistency of interpretation, the subspecialty with the fewest median manuscript citations was selected as a reference in multivariable models.
Finally, the relationship between citations and author Twitter account details was assessed for both first and senior authors. Twitter account details were compared between first and senior authors, including the association between H-index and Twitter followers. The ratio of Twitter followers to H-index was compared among subspecialties [20].

Results
After exclusion criteria were applied, 4,224 manuscripts remained, which had been cited a total of 53,635 times. Of the articles, 62.1% were mentioned online, and 94.2% of these (58.5% of the total) were mentioned on Twitter (  Categorical variables are reported as n (%) where the % is in reference to the column total. Continuous variables are reported as median (interquartile range), except journal impact factor which is reported as value (standard deviation). Boldface p-values denote statistical significance (p < 0.05).

Manuscript-Level Analyses
There were significant differences between subspecialties in both citations and Twitter mentions ( Figure 1). Articles in sports medicine-related journals were cited the most frequently (median 11, IQR 6-20), while articles in paediatrics (median 5, IQR 2-9) and trauma (median 5, IQR 3-9)-related journals were cited least frequently ( Figure 1). Similarly, articles published in sports medicine journals were mentioned on Twitter most frequently (72.5% of articles) and articles published in paediatrics journals were mentioned on Twitter least frequently (26.3% of articles, Figure 1). Articles published in open-access journals received fewer citations (median 9 vs. 6, p < 0.0001, Table 3).  In multivariable analysis over the full set of articles, the number of Twitter mentions ( β est = 0.0055, p < 0.0001), journal impact factor (β est = 0.14, p < 0.0001), and non-open access status ( β est = 0.35, p < 0.0001) were associated with greater citation count. For purposes of interpretation, the log of the number of citations was observed to increase by 0.0055 with each additional tweet mentioning the article. For context, 10 tweets would be associated with a 1.1-fold increase in citations, 100 would be associated with a 1.7-fold increase, and 1,000 tweets would be associated with a 245-fold increase. Compared to articles published in pediatrics-related journals, articles published in all other subspecialty-related journals except trauma received more citations ( Table 4).

Author-Level Analyses
A random selection of 20% of the manuscripts (n=780) was selected for author-level analyses after exclusion. Three hundred and twenty (41%) of these manuscripts featured either a first or senior author with a Twitter account, with 156 first-author accounts and 216 senior-author accounts ( Table 5). There were significant differences in rates of author Twitter presence by journal subspecialty, with sports medicine-related journals featuring a Twitter author most frequently (53.4%) and foot and ankle-related journals least frequently (14.6%, Figure 2). Manuscripts with at least one author on Twitter were cited more frequently than those without either author on Twitter (median 10 vs. 8, p < 0.001, Figure 3). Articles with both first and senior authors on Twitter were cited most frequently (median 15.5, IQR 7-25).

TABLE 5: Summary of a random sample of 780 articles selected for author-level analyses
Categorical variables are reported as n (%) where the % is in reference to the column total. Continuous data are reported as median (interquartile range). Boldface p-values denote statistical significance (p < 0.05). UE: upper extremity.

FIGURE 2: Percentage of articles having at least one of first or senior author with a Twitter account
Overall Kruskal Wallis p-value is <0.0001.

FIGURE 3: Manuscript citations by author Twitter presence
Data are presented as the natural log of one plus citation count in order to avoid taking the log of zero. Violin plots are scaled according to a Gaussian density approximation. Overlying boxplots depict median and interquartile range.

TABLE 7: Twitter follower to H-index ratio among subspecialties
Presented as median and interquartile range. IQR: interquartile range.

Discussion
Although only a quarter of authors had a Twitter account, those who did received more citations, even after controlling for subspecialty, journal impact factor, H-index, and open access status. There were significant differences by subspecialty in citations, Twitter mentions, author Twitter utilization, and the ratio of Twitter followers to H-index. This is the first study to consider author-level Twitter utilization in relation to both subspecialty and citation count and is also the largest study in the orthopaedic surgery literature on the topic of factors correlating with citations.
This study identified the author's Twitter presence as an independent predictor of increased citations. While the mean H-index of the top 72 most influential orthopaedic surgeon Twitter accounts has been reported to be 13.7 [9], the association between Twitter utilization and manuscript citations has not previously been described. Interestingly, Tweet count and Twitter follower count correlated with citations more strongly for first compared to senior authors, suggesting Twitter activity may be more strongly related to citation count for those earlier in their careers. Younger researchers, who have fewer publications, have likely attended fewer conferences, and likely have smaller professional networks, may rely more heavily on online platforms for networking than their more senior peers. In contrast, citations for senior authors were more closely related to academic metrics, such as H-index.
Regarding the influence of subspecialty, articles published in sports medicine, general interest, and arthroplasty journals were cited more, with sports medicine journal manuscripts being cited most (median 11,. This corroborates a recent study that found arthroplasty, oncology, and sports medicine to be the three highest-cited orthopaedic subspecialties [21]. Additionally, sports medicine journals had the highest percentage of authors on Twitter (53.4% overall) and the highest percentage of manuscripts being mentioned on Twitter (72.1%). Factors contributing to the higher citation count may include the competitiveness of the specialty as well as the degree of specialization and the number of journals. Despite having the highest citation count, sports medicine-related authors also had one of the highest Twitter follower-to-H-index ratios, suggesting their accounts draw even more attention than would be expected for a given H-index. Sports medicine surgeons may be more likely to engage in online activity related to topics of popular interest (e.g., professional athlete injuries), which could increase their follower count. Arthroplasty-related journals may also receive more citations through a similar mechanism. It is expected that general interest journals (JBJS, JAAOS, and CORR) receive greater attention given their larger audience.
While Twitter previously accounted for 82% of the online article mentions in orthopaedics [1], the present study found that Twitter now accounts for over 90%, indicating the platform may be increasing its share of orthopaedic surgery-related social media utilization. Twitter utilization in orthopaedics may be similar to other medical specialties. Our estimate of 58.5% of articles mentioned on Twitter is lower than 73% in urology [22] but higher than 50.7% in otolaryngology [23]. Regarding effect size, urology and otolaryngology articles mentioned on Twitter received 2.0-fold and 1.6-fold more citations than those not mentioned, respectively. In contrast, the present study found a 1.3-fold increase in citations for articles mentioned on Twitter (median 10 vs. 7). Importantly, these authors approximated effect sizes using mean citation counts [22,23], which are highly skewed, and estimates based on the median would likely be lower.
The study corroborated the findings of prior studies by demonstrating positive correlations between Twitter mentions (rho = 0.25) and AAS (rho = 0.31) and citations. In 2017, Evanvew et al. analyzed the relationship between AAS and citations for 1,675 randomized controlled trials and estimated the rho to be 0.11 [1]. In 2020, Zhang and Earp estimated rho to be 0.10 for AAS and 0.11 for Twitter mentions for 835 articles from JAAOS, JBJS, and CORR [21]. Also in 2020, Kunze et al. estimated rho to be 0.31 for AAS in a sample of 496 articles from seven journals [18]. Reasons for the larger correlation in the present study compared to the prior studies include the larger sample of manuscripts from a range of journals and manuscript types, the increasing time between article publication and analysis, and the increasing utilization of Twitter.
Interestingly, articles published in open-access journals received fewer citations, even when controlling for the impact factor. Prior literature on the impact of open access status has been conflicting. For example, Silva et al. identified a weakly positive association among sports medicine articles [5], but an inverse relationship has also been identified in other fields [19]. Overall, a weak effect of open access status and citations may be due to the prevalence of institutional availability of non-open-source articles. Furthermore, open-access journals are relatively young compared to other established subscription journals, which may contribute to lower citations or a lower preference for authors to cite articles from those journals.
The strengths of this article include the incorporation of author-level social media utilization, a large sample size of manuscripts, and the inclusion of a variety of journals representing different subspecialties. There are several limitations to our analysis of the author Twitter profiles, however. For example, some authors may run pseudonymous Twitter accounts that cannot be identified. The content of the author's tweets may also be unrelated to medicine. The exact mechanism of the Twitter feed-generating algorithm is not known and may bias the presentation of certain articles. Finally, an author's propensity for Twitter utilization may not be independent of other personal or professional factors related to academic productivity. Additionally, aside from applying exclusion criteria to certain types of articles, this study did not adjust for specific study design factors such as sample size or risk of bias. Finally, journal-specific Twitter activity and other factors may also be associated with citations [6,24]. Ultimately, it is not possible to determine causation in this retrospective analysis; it is not clear if articles mentioned on Twitter ultimately reach a higher impact as measured by citations or if higher-impact articles are more likely to be shared on social media. Other social media sectors not analyzed, such as Facebook, LinkedIn, and orthopaedic-specific platforms like AOC Connect and Ortho Social Media Platform, may also have impacts beyond Twitter and warrant future investigation to understand the ideal media for dispersing knowledge and research.
Author Twitter utilization is associated with academic citation count independently of other author and manuscript factors. Future studies should examine the content of tweets, the scientific quality of manuscripts, and the Twitter activity of specific journals. As research dissemination rapidly evolves online, authors should be aware of the impact of social media utilization on academic impact in order to effectively share findings and assess research impact.

Conclusions
Twitter utilization and manuscript citations are independently associated with authors who publish in orthopaedic journals. Those who appreciate this potential link between social media implementation and traditional academic impact can benefit from more effective dissemination of their research. Additionally, understanding this association is essential for appraising the impact of evolving online platforms for research circulation. Future studies should examine Twitter activity within specific institutions and journals, the content of tweets, and the scientific quality of the manuscripts shared on the platform.

Additional Information Disclosures
Human subjects: All authors have confirmed that this study did not involve human participants or tissue. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue.

Conflicts of interest:
In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.