A Situational Analysis of the Impact of the COVID-19 Pandemic on Digital Health Research Initiatives in South Asia

The objective of this paper was to evaluate and compare the quantity and sustainability of digital health initiatives in the South Asia region before and during the COVID-19 pandemic. The study used a two-step methodology of (a) descriptive analysis of digital health research articles published from 2016 to 2021 from South Asia in terms of stratification of research articles based on diseases and conditions they were developed, geography, and tasks wherein the initiative was applied and (b) a simple and replicable tool developed by authors to assess the sustainability of digital health initiatives using experimental or observational study designs. The results of the descriptive analysis highlight the following: (a) there was a 40% increase in the number of studies reported in 2020 when compared to 2019; (b) the three most common areas wherein substantive digital health research has been focused are health systems strengthening, ophthalmic disorders, and COVID-19; and (c) remote consultation, health information delivery, and clinical decision support systems are the top three commonly developed tools. We developed and estimated the inter-rater operability of the sustainability assessment tool ascertained with a Kappa value of 0.806 (±0.088). We conclude that the COVID-19 pandemic has had a positive impact on digital health research with an improvement in the number of digital health initiatives and an improvement in the sustainability score of studies published during the COVID-19 pandemic.


Introduction And Background
The Global Digital Health Strategy 2020-2025 of the World Health Organization (WHO) defines digital health as the field of knowledge and practice associated with the development and use of digital technologies to improve health [1].The fit between individuals, task, and technology (FITT) framework developed by Ammenwerth et al. [2] for the classification of information technology initiatives broadly classified digital health initiatives into three tasks: health information exchange (HIE), data science, and surveillance [3].The WHO's Global Digital Strategy stated its vision as "improving health for everyone, everywhere by accelerating the development and adoption of appropriate, accessible, affordable, scalable, and sustainable person-centric digital health solutions."The vision document has highlighted four strategic objectives: (a) institutionalization of digital health in the national health system, (b) integration of strategy to ensure the success of digital initiatives, (c) promotion of appropriate use of digital technologies for health, and (d) addressing the impediments faced by least developed countries implementing digital health technologies.
The COVID-19 pandemic disrupted the global healthcare systems, the increasing COVID-19 cases diverted the available healthcare infrastructure, resources, and manpower toward controlling the pandemic, and the delivery of normal healthcare services related to maternal care and childcare and non-communicable diseases was disrupted [4][5][6][7].The disruption in the normal delivery of healthcare services presented an opportunity for digital health initiatives to fill in the gap, with as many as 264 research papers on digital health indexed in PubMed in the year 2020 from the South Asia region as compared to 82 research papers indexed in the year 2019.The threefold increase in Medline-indexed research papers implies an increased interest in medical practitioners, researchers, and academicians in digital health from developing countries such as India, Bangladesh, Vietnam, Thailand, Sri Lanka, and Indonesia.

Descriptive Analysis of the Number of Research Articles Published on Digital Health
Our search strategy yielded a total of 563 articles on Medline.With a primary screening of article titles and abstracts, 186 articles were found to satisfy the eligibility criteria.From 2016 to 2020, the number of research articles published on digital health has been steadily increasing, and the trend over the past five years is visualized in Figure 2. When the studies were stratified based on study design, review articles and guidelines made up 65% of all the studies published on digital health in the past five years, with a maximum number of review articles published in 2020.Figure 3 visualizes the study design and year trend.The year-on-year trend presented an average of 10% increase in review articles, a 5% increase in letters to editors and correspondence, a 4% increase in diagnostic studies, and a 2%-3% increase in experimental studies, cohort studies, pilot studies, and development studies on digital health between 2016 and 2021.

FIGURE 3: Design-wise and year-wise study stratification
The studies were stratified based on diseases and conditions or applications for which digital initiative was primarily targeted across the five-year timeline (Figure 4).Health systems strengthening, which included initiatives such as capacity building among healthcare staff, geospatial analysis for improving access to healthcare services, and the use of data science to improve the availability of essential medicines, was the area wherein the highest digital health initiatives were focused.Ophthalmic disorders and COVID-19 were the second and third conditions to be frequently researched.For the year-on-year trend analysis of conditions or applications wherein digital health initiatives were primarily targeted, ophthalmic disorders and health systems strengthening recorded a <20% increase and cancer screening, cardiovascular disorders, and mental health recorded a 4%-6% increase on average from 2016 to 2021.

FIGURE 4: Stratification of studies on diseases and conditions or applications
COVID-19: coronavirus disease 2019, HIV: human immunodeficiency virus A negative year-on-year trend was seen for dental sciences, dermatology, drug safety, elderly care, HIV prevention and control, injuries and accidents, kidney and urinary tract disorders, leprosy management, neurological conditions, nutrition, orthotic disorders, sexual and reproductive health, stroke, tobacco and alcohol cessation, tuberculosis prevention and control, vaccination, and vector-borne disease prevention and control.
The study used the FITT framework to classify digital tools on tasks.The broad classification was health information exchange, data science, and surveillance.Further classification and definition with examples of each digital tool are presented in Table 2.

HIE Data sharing
Any tool a platform has to support data sharing between heterogeneous computer systems of different organizations [15] Health information and management system

Information delivery
Tool used for the delivery of information (preventive, curative) to the end users SMSs sent for reminders of upcoming health visits

Remote consultation
Consultation by remote telecommunications, generally for the purpose of the diagnosis or treatment of a patient at a site remote from the patient or primary physician [16] Consultation using Skype and/or Zoom

Intelligent diagnosis
An intelligent diagnosis system is one that is capable of identifying the nature of a problem by examining the observed symptoms and possibly an explanation or justify the same [17] Algorithm-based diagnosis of risk of developing diabetes

Data science
Patient-generated health data Health-related data created, gathered, or inferred by or from patients and for which the patient controls data collection and data sharing [18] Digital diary

Predictive analytics
Predictive analytics is a branch of advanced analytics that makes predictions about future outcomes using historical data combined with statistical modeling, data mining techniques, and machine learning [19] Predictions on chances of raining

Clinical decision support system
Tools designed to be a direct aid to clinical decision-making, in which the characteristics of an individual patient are matched to a computerized clinical knowledge base and patient-specific assessments or recommendations are then presented to the clinician for a decision [20] Image-based computer-assisted screening of oral lesions for cancer screening Big data mining Big data analytics covers the integration of heterogeneous data, data quality control, analysis, modeling, interpretation, and validation [21] Using air pollution data to develop Air Quality Index

Surveillance
Risk screening Continuous risk assessment of a condition or population through multiple screening surveys [22] NA

Real-time data collection and visualization
Ongoing collection of data and presentation of data in a pictorial or graphical format [23] COVID-19 dashboards Contact tracing Digitalization of identification and follow-up of persons who may have come into contact with a person infected with the infectious diseases [24] NA The studies on remote consultation were most commonly reported, followed by information delivery systems and clinical decision support systems, as visualized in Figure 5.The year-on-year trend analysis of the digital tools used presented an average annual increase of 5% in health information exchange, with remote consultation recording a 9% average annual increase, followed by a 7% increase in health information delivery.Data science tools recorded a 0.5% annual increase, and under data science tools, only clinical decision support systems recorded a 2.5% average annual increase, whereas big data mining, predictive analysis, and patient-generated health data tools recorded a negative year-onyear trend.The surveillance tools usage trend presented a 0.5% average annual increase for 2016-2021, with real-time data collection and visualization tools showing a 2% increase.
The descriptive analysis highlights the following: (a) the COVID-19 pandemic has positively impacted digital health research with a 40% increase in the number of studies reported in 2020 when compared to 2019, (b) the three most common diseases and conditions wherein substantive digital health research has been focused are health systems strengthening, ophthalmic disorders, and COVID-19, and (c) remote consultation, health information delivery, and clinical decision support systems are the top three commonly developed tools from 2016 to 2021.

Assessment of Sustainability
Validation of sustainability assessment tool: We use inter-rater operability as an indicator to measure the ease of replicability of the tool.An independent rater assessed 30 randomly selected studies using a computer-generated list of the 87 articles assessed in section B of this study.Cohen's unweighted Kappa was used to determine the inter-rater operability.The overall Kappa value was 0.806 (±0.088).The highest interrater operability was seen for Q1 (Does the study involve a multisectoral team?), with a Kappa value of 1 (100%), and the lowest inter-rater operability was seen for Q3 (Does the study mention adherence to any standards of data components, data interchange, and application-level support?), with a Kappa value of 0.54 (±0.036).
The Kappa values for the other questions were as follows: (a) Q2 (Does the study involve engagement with sectors other than health?), 0.91 (±0.066);(b) Q4 (Does the study mention stakeholder analysis/community needs assessment/with end users for the development of initiatives?), 0.84 (±0.042); and (C) Q5 (Does the study mention the scope of collecting feedback from the end users?),0.71 (±0.06).
The interpretation of results was divided into studies published before the COVID-19 pandemic (2016-2019) and during the COVID-19 pandemic (2020-2021).Analysis was reported for the study design.

Experimental Study Design
All experimental studies before and during the pandemic had an active involvement of the IT and data management teams while developing the intervention and were either part of the team writing the manuscript or fully acknowledged, indicating that the multisectoral team was involved in developing and testing the initiatives.
Ten of 18 experimental studies involved the engagement of multiple sectors; for example, Swendeman et al. [25] included behavioral scientists, HIV care providers, and frontline health workers in the implementation of the study.
All the experimental studies mentioned adherence to data standards such as WHO or International Classification of Diseases (ICD) standards; however, adherence to data interchange standards such as HL7 was not mentioned.
All the experimental studies have either conducted gap analysis or referred to previously published authors' papers on gap analysis and community needs assessment for the development of initiatives.All the experimental studies have mentioned feedback collection from end users and delivery providers and have mentioned changes made in digital initiatives upon receiving feedback.
Overall, the average sustainability of experimental studies on digital health was 80%, and there was no statistically significant difference in overall sustainability score between the studies published prepandemic (85.6%) and during the pandemic (76.4%) (p=0.33).
Table 3 and Figure 7 present the study-wise assessment score summary and percentage of sustainability scores based on the authors' judgment.(

Cohort Study Design
Of the cohort studies, 80% had active involvement of the IT and data management teams while developing the intervention and were either part of the team writing the manuscript or fully acknowledged, indicating that the multisectoral team was involved in developing and testing the initiatives.The pattern of engagement with multisectoral was similar in studies reported before the COVID-19 pandemic and during the COVID-19 pandemic.However, 80% of the studies assessed did not show engagement of multiple sectors.
Of the cohort studies, 65% reported adherence to WHO/International Classification of Diseases (ICD) standards for the classification of diseases; however, studies have not mentioned adherence to data interchange standards such as HL7.
All the cohort studies assessed have mentioned gap analysis and needs assessment for the development of initiatives.About 60% mentioned feedback collection from end users and delivery providers and mentioned changes made in digital initiatives upon receiving feedback.
Overall, the average sustainability of cohort studies on digital health was 40%, and there was no statistically significant difference in overall sustainability score (if the score has been determined and the tool's validity has been assessed, this information is reported in a yes or no format) between the studies published prepandemic (25.3%) and during the pandemic (35.6%) (p=0.45).
Table 4 and Figure 8 present the study-wise assessment score summary and percentage of sustainability scores based on the authors' judgment.(

Cross-Sectional Study Design
Of the cross-sectional studies, 25% had active involvement of the IT and data management teams while developing the intervention and were either part of the team writing the manuscript or fully acknowledged, indicating that the multisectoral team was involved in developing and testing the initiatives.The percentage of active involvement of the multisectoral team was higher in cross-sectional studies reported during COVID-19 as compared to studies reported pre-COVID-19.However, in 80% of cross-sectional studies assessed, engagement of multiple sectors was not seen.
Of the cross-sectional studies reported during the COVID-19 pandemic, 90% reported adherence to WHO/ICD standards for the classification of diseases, whereas only 10% of cross-sectional studies reported pre-COVID-19 reported adherence to WHO/ICD standards for the classification of diseases.
Of the cross-sectional studies, 30% have mentioned gap analysis and needs assessment for the development of initiatives.About 40% of the studies have mentioned feedback collection from end users and delivery providers and have mentioned changes made in digital initiatives upon receiving feedback.
Overall, the average sustainability of cross-sectional studies on digital health was 40%, and there was a statistically significant difference in overall sustainability score between the studies published prepandemic (45.3%) and during the pandemic (27.7%) (p=0.002).
Table 5 and Figure 9 present the study-wise assessment score summary and percentage of sustainability scores based on the authors' judgment.

Diagnostic Studies
Of the diagnostic studies, 70% had active involvement of the IT and data management teams while developing the intervention and were either part of a team writing the manuscript or fully acknowledged, indicating that the multisectoral team was involved in developing and testing the initiatives.The percentage of active involvement of multisectoral teams was higher in diagnostic studies reported during the COVID-19 pandemic compared to studies reported pre-COVID-19 pandemic.However, in 95% of diagnostic studies assessed, engagement of multiple sectors was not seen.
Of the diagnostic studies, 60% reported adherence to WHO/ICD standards for the classification of diseases; the majority of studies reported during the COVID-19 pandemic were able to provide information on adherence to standards.
Of the diagnostic studies assessed, 25% have mentioned gap analysis and needs assessment for the development of initiatives.About 30% of the studies have mentioned feedback collection from end users and delivery providers and have mentioned changes made in digital initiatives upon receiving feedback.
Overall, the average sustainability of diagnostic studies on digital health was 45%, and there was no statistically significant difference in overall sustainability score between the studies published prepandemic (45%) and during the pandemic (55%) (p=0.5) Table 6 and Figure 10 present the study-wise assessment score summary and percentage of sustainability scores based on the authors' judgment.

Pilot Studies and Development Studies
We used the definition given by Stewart [92] for pilot and development studies as a "small study to test research protocols, data collection instruments, sample recruitment strategies, and other research techniques in preparation for a larger study."However, pilot randomized studies were included in experimental studies.
Of the pilot and development studies, 75% had active involvement of the IT and data management teams while developing the intervention and were either part of the team writing the manuscript or fully acknowledged, indicating that the multisectoral team was involved in developing and testing the initiatives.The percentage of active involvement of multisectoral teams was similar for studies reported during the COVID-19 pandemic and studies reported before the COVID-19 pandemic.However, in 95% of diagnostic studies assessed, engagement of multiple sectors was not seen.
Of the pilot and development studies, 80% reported adherence to WHO/ICD standards for the classification of diseases.
Of the pilot and development studies assessed, 50% have mentioned gap analysis and needs assessment for the development of initiatives.Similarly, 50% of the studies have mentioned feedback collection from end users and delivery providers and have mentioned changes made in digital initiatives upon receiving feedback.
Overall, the average sustainability of pilot and development studies on digital health was 65%, and there was no statistically significant difference in overall sustainability score between the studies published prepandemic (56%) and during the pandemic (63%) (p=0.28).
Table 7 and Figure 11 present the study-wise assessment score summary and percentage of sustainability scores based on the authors' judgment.

Retrospective Analysis Studies
Three studies were assessed for sustainability using retrospective analysis study design as shown in

Discussion
To our knowledge, this study is the first to systematically analyze and present evidence from the sustainability perspective of digital health research initiatives across South Asia.A systematic review by Bassi et al. [111] in 2018 looked at the current status and future perspectives of mHealth from health systems perspectives; the review highlighted the poor quality of evidence generated through mHealth research.
Another review by Bassi et al. [112] in 2020 presented the review of COVID-19-related mobile applications and highlighted gaps to inform the development of future mHealth initiatives, wherein the functionality of mobile applications was assessed to adjunct risk assessment efforts.Another systematic review by Kondylakis et al. [113] raised concerns about the quality of research studies published on the development and implementation of COVID-19 mobile applications.
The COVID-19 pandemic saw a huge increase in research on digital health initiatives; our study hypothesis was based on the increased number that has affected the sustainability of the research initiatives and the intrinsic factors of research initiatives such as multisectoral involvement, gap analysis, and stakeholder engagement.
In this paper, we descriptively analyzed the impact of the COVID-19 pandemic on the volume of research on digital health.To ascertain sustainability, we searched for standardized tools available for sustainability assessment; however, the tools did not suit the needs presented in this study.This led to the development of a simple tool for the assessment of sustainability.The assessment of sustainability was undertaken for 87 articles, and the tool can determine intrinsic sustainability factors and give summary estimates on how well the authors incorporate sustainability in digital health research.
In our study, we found that the number of digital health intervention research increased significantly during the COVID-19 pandemic, and most authors took the initiative to have oversight on the sustainability of their digital health initiatives.Contrary to our hypothesis, the sustainability score of cross-sectional studies was higher for studies published during the COVID-19 pandemic as compared to studies published before the COVID-19 pandemic.
According to our study findings, it is clear that the choice of digital tool, disease, and study design vary highly based on study objectives and research; however, the patterns presented over five years show higher research interest in ophthalmic disorders and health systems strengthening, while health information exchange tools have been exclusively used.The underlying reasons can be further explored and taken up for further research and help derive recommendations at the policy level.
a) Were all the authors from the medical field only, and (b) are the data scientists/development engineers/IT consultants acknowledged in the papera) With respect to the study objective, did the study mention any other sectors such as nutrition or agriculture, and (b) were the study endpoints or outcome assessment involved data from other sectors?application-level support?Consider: (a) Data components -reference information model, data elements, data types, and terminology (standards: HL7, SNOMED, LOINC, and UMLS), (b) data interchange -structured and free form documents and images (standards: HL7 and ASTM), and (c) application-level support -disease registries and implementation manuals (standards: HIPAA, HL7, ASTM, a) Does the study cite previous studies published on gap analysis, stakeholder analysis, and qualitative surveys to understand the needs of the end users and (b) does the study involve a section or subsection wherein end users were involved in the development of a digital health initiative?Yes, no, can't say (if your answer is yes, +; no, -1; and can't say, 0) Q5: Does the study mention the scope of collecting feedback from the end users?Consider: (a) Was there any mention of the collection of feedback from the end users in the paper, and (b) in the case of applications, has feedback been collected and suitable changes been made to improve user-friendliness?

FIGURE 1 :
FIGURE 1: PRISMA flow diagram for the selection of articles for the study PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

FIGURE 2 :
FIGURE 2: Year-wise trend in the number of publications Note: For the year 2021, data was collected only for the first six months as the data collection phase ended mid of 2021.

FIGURE 5 :
FIGURE 5: Stratification of studies based on digital tool used DS: data science, HIE: health information exchange, SUR: surveillance

TABLE 2 : Types of digital tools
HIE: health information exchange, SMS: short message service

Table 8
and Figure 12.Kammari et al. conducted a study pre-COVID-19 pandemic that demonstrated 80% chances of sustainability.