Transforming Healthcare: The Role of Artificial Intelligence, Machine Learning, and Health Systems

Globally, health systems grapple with significant pressures: an increasing burden of illness, multimorbidity, and disability fueled by aging populations and changing disease patterns; escalating demand for services; heightened societal expectations; and rising expenditures [1]. Compounding these issues is systemic inefficiency and poor productivity [2]. These challenges persist amidst fiscal conservatism, where often misplaced economic austerity measures hinder necessary investment in health systems. A fundamental transformation is urgently needed to overcome these hurdles and achieve Universal Health Coverage (UHC) by 2030. It is within this context that artificial intelligence, machine learning, and health systems emerge as potential game-changers. Machine learning, the most prominent application of AI and a rapidly growing area in digital technology, holds the promise of achieving more with less, potentially catalyzing this essential transformation [3]. However, the true scope and nature of this promise require systematic assessment. Historically, the impact of digital technology on health systems has been ambiguous [4]. Will AI be the key ingredient for transformation, or will it meet the same fate as previous digital initiatives? This article explores the potential applications of AI within health systems and examines how it could reshape them to achieve UHC by enhancing the efficiency, effectiveness, equity, and responsiveness of public health and healthcare services.

Understanding AI and Machine Learning in Healthcare

Artificial Intelligence (AI) is a vast field dedicated to designing systems exhibiting intelligent properties, notably the ability to learn from data [6]. While this broad definition overlaps with existing statistical methods, the recent surge in progress stems primarily from a subset of AI: machine learning, particularly deep learning.

Defining Key Concepts

• Artificial Intelligence (AI): A broad scientific discipline rooted in philosophy, mathematics, and computer science, focused on understanding and creating systems that display intelligent behaviour.
• Machine Learning (ML): A subfield of AI where computer algorithms learn predictive associations from data examples. It applies statistical models to data using computation, often employing a wider range of techniques than traditional medical statistics. Newer methods like Deep Learning handle complex data with fewer assumptions [5].
• Deep Learning: A class of ML methods allowing machines to process large amounts of raw data (like images) and discover patterns needed for detection or classification. It uses multiple layers of data representation, amplifying important features and suppressing irrelevant ones. Deep learning can be supervised or unsupervised and has driven many recent foundational ML advances [5].
• Supervised Learning: Training algorithms by providing labeled data (inputs and corresponding desired outputs defined by humans). The learned associations are then used to predict outcomes for new, unseen data. This is well-established in healthcare and beyond.
• Unsupervised Learning: Algorithms learn patterns and associations within data without predefined outputs. This approach can identify previously unknown predictors.
• Reinforcement Learning: Algorithms learn actions by maximizing a defined reward, influenced by behavioural psychology. It excels in environments with perfect information, numerous options, and no real-world failure cost, like games.

The Rise of Deep Learning

Deep learning systems have achieved remarkable success, setting new performance benchmarks in various economic sectors where high-quality digital data abound and automation offers strong incentives [5]. In these contexts, deep learning algorithms, guided by human expertise for data curation and algorithm tuning, have successfully uncovered valuable predictive associations, typically for specific tasks [7]. It’s crucial to understand that these advancements in machine learning do not represent “artificial general intelligence” – a versatile, human-like intelligence capable of independent cross-domain application and self-reinforcing concept integration.

How Artificial Intelligence and Machine Learning Are Reshaping Health Systems

Effective health system management, much like public health or clinical care delivery, fundamentally relies on a network of information processing tasks. Policymakers adjust health system functions—organization, governance, financing, resource management—to influence outputs (services, public health) and achieve system goals [8].

Enhancing Decision Making

Healthcare provision itself involves two primary information processing tasks:

1. Screening and Diagnosis: Classifying cases based on history, examination, and investigations.
2. Treatment and Monitoring: Planning, implementing, and tracking multi-step processes to achieve future outcomes.

Across both health system management and care provision, the core process involves hypothesis generation, testing, and action. Machine learning holds significant potential to enhance hypothesis generation and testing by uncovering previously hidden patterns in data. This capability can profoundly impact both individual patient care and system-level operations.

Machine learning expands upon traditional statistical techniques [6]. By using methods not bound by prior assumptions about data distribution, it can identify complex patterns that inform hypothesis formulation and testing. While ML models can be harder to interpret than traditional statistics, they can incorporate far more variables, handle diverse data types, and yield results in complex scenarios [9].

Key Application Areas

Research has already demonstrated ML applications in screening, diagnosis, and predicting future events, although often in hospital settings and based on single-center data, raising questions about reproducibility [11] and generalizability [12]. Despite these limitations, the rapid evolution of machine learning continues across all sectors [13]. Examples include:

• Waveform Analysis: Monitoring fetal heart rate during labor (s31), remote gait monitoring in neurology (s35).
• Image Analysis:
  • Pathology: Detecting lymph node metastases in breast cancer (s37), predicting colorectal cancer outcomes (s38).
  • Dermatology: Identifying benign/malignant tumors (s39), fungal infections (s40), classifying skin cancer (s41).
  • Ophthalmology: Identifying diabetic retinopathy (s43), grading macular degeneration (s44).
  • Cardiology: Diagnosing acute coronary syndrome (s46), remote monitoring of heart failure status (s47).
  • Radiology: Improving mammography interpretation (s49), diagnosing pneumonia from chest X-rays [10].
• Electronic Health Record (EHR) Analysis: Predicting inpatient diagnoses (s52), identifying sepsis in the emergency department (s53), spotting breast cancer symptoms (s54), identifying heart failure cases (s55), analyzing ICU data for patient phenotypes (s56), extracting ICD-10 codes from death certificates (s58).
• Prognosis and Prediction: Cardiovascular risk prediction (s32-34), predicting breast cancer survival (s36), non-small cell lung cancer survival (s42), hospitalization due to heart disease (s45), primary care utilization (s48), sepsis development (s50), central line infections (s51), treatment outcomes in social anxiety (s59), psychiatric readmission (s60).
• Claims Analysis: Screening for type 2 diabetes from payer data (s61).

(References s31 to s61 are available in the Online Supplementary Document)

Current Limitations

While promising, current applications often rely on data from single institutions, which may limit how well these models perform in different settings or populations [11, 12]. Ensuring broad applicability and reliable performance across diverse healthcare environments remains a key challenge.

The Impact of AI and Machine Learning on Clinical Care and the Health Workforce

Machine learning has evolved into a “General Purpose Technology” (GPT) – it’s pervasive, continuously improvable, and capable of spurring complementary innovations [14]. Historically, the rollout of GPTs often causes “widespread economic disruption, with concomitant winners and losers” [15].

Automation and the Displacement Effect

Economists Acemoglu and Restrepo analyzed automation’s historical impact, identifying a displacement effect where machines replace human labor in tasks where they hold an advantage [16]. However, this is counterbalanced by forces that increase labor demand: a productivity effect arises as operations become cheaper and more efficient. Savings can then be reinvested into existing non-automatable tasks or used to create new ones, including roles managing the automation technology itself.

Case Study: Diagnostic Radiology Transformation

To understand how these dynamics might affect the healthcare workforce, consider diagnostic radiology, an area heavily featured in ML literature. As deep learning algorithms demonstrate superior performance in image analysis, some predict the decline of radiologists, even questioning the need to train new ones [17].

It’s plausible that ML will initially allow existing radiologists to manage higher caseloads. Eventually, as systems become more autonomous, some diagnostic image analysis tasks could shift to non-radiologists supported by ML tools. This task reorientation presents an opportunity for health systems to adjust the skill mix and distribution of radiology teams. More routine tasks might move to primary care settings, while complex, non-automatable work and rare cases remain with a smaller cohort of specialized radiologists in secondary/tertiary centers.

For instance, researchers behind an ML system for pneumonia diagnosis [18] developed a tool where the AI first “reads” the image, highlighting areas of concern for the human radiologist. This improves workflow efficiency by directing limited human attention effectively, enabling radiologists to handle more cases [10]. Similar transformations are expected in pathology and other image-reliant specialties [19,20].

The Future: Human-Computer Collaboration

Machine learning will likely foster processes performed by human-computer hybrids. This synergy offers the potential to optimally combine human strengths—hypothesis generation, collaboration, oversight—with AI’s ability to analyze vast datasets for predictive patterns or optimization. Jha and Topol suggest merging radiology and pathology into a new “information specialist” role. These specialists would focus less on primary information extraction (done by AI) and more on managing AI-generated information within the patient’s clinical context (reference s21 in Online Supplementary Document).

Healthcare professional analyzing medical data on multiple screens, illustrating human-computer collaboration in AI-driven health systems.

Implications for Skill Mix and Service Delivery

This shift could significantly enhance the quality and scope of care, potentially without drastic cost increases, especially if similar amalgamations occur in other specialties and more tasks shift towards primary care.

Building a Receptive Environment for AI in Health Systems

Machine learning is advancing rapidly. However, like any new technology, its successful integration into health systems depends not just on technical prowess but also on establishing a receptive context for adoption and diffusion (s22, s23). Key elements of this context include:

The Crucial Role of Data Curation and Interoperability

A long-standing challenge in health IT is interoperability – the lack of common data standards hinders data aggregation across health systems. Current approaches often fail existing data needs and must be re-evaluated to accommodate the greater demands of machine learning (s24).

Encouragingly, an unpublished study by Google and three academic medical centers demonstrated combining data from different hospitals without prior conversion to a common format. Deep learning applied directly to native-format data outperformed existing benchmarks in predicting mortality, readmission, length of stay, and diagnoses (s25). While such advances might reduce the need for complete data harmonization, they don’t eliminate it. Health systems must still prioritize initiatives that aggregate data for both current functions and future ML applications. The potential impact of artificial intelligence, machine learning, and health systems integration should galvanize these efforts.

Addressing Algorithmic Bias and Ensuring Equity

Machine learning models, while potentially technically superior, won’t be perfect. Some individuals will inevitably be negatively affected. There’s a significant risk that these individuals will belong to marginalized groups underrepresented in the datasets used for algorithm training. Consequently, ML could inadvertently worsen existing health inequities. To mitigate this “algorithmic bias” (s26), it’s crucial to ensure diverse representation in datasets, use data from various clinical sites, and promote diversity among algorithm developers. Without these safeguards, social inequities observed elsewhere could be replicated or amplified in healthcare.

Building Trust Through Data Governance and Transparency

A major barrier to creating the large datasets needed for ML development is a lack of public trust regarding data usage. High-profile instances of data sharing transgressing legal safeguards have further damaged the relationship between citizens (data sources) and data users (s27).

Unlike the consumer digital economy, where users implicitly trade data for services (better search results, relevant feeds), patients may not perceive a similar value proposition in healthcare data sharing. Concerns about trust in government, privacy violations, and potential discrimination based on health status often outweigh the perceived distal benefits of sharing health data.

Collaborating with the Technology Industry

The reality is that major ML advancements often originate from, or require collaboration with, a few large technology companies that possess the necessary capital, computing resources, and expertise. This concentration, however, raises concerns about economic power consolidation and creates legislative complexities for governments potentially becoming reliant on private entities for core AI infrastructure.

Therefore, innovative contracting mechanisms are needed to facilitate collaboration with these companies. These frameworks must enable national-scale health data capture and use while rigorously protecting privacy and ensuring fair attribution of resulting intellectual property. Currently, no standard agreement exists, presenting a critical opportunity for international public health organizations to provide leadership and for corporations to demonstrate social responsibility, ensuring AI’s benefits are shared broadly. A wider dialogue involving citizens, health system data custodians, and private tech companies is essential to resolve issues around intellectual property, the public good vs. private capital nature of health data, and patient privacy concerns.

Ensuring Accountability and Explainability

Machine learning systems, especially deep neural networks, often function as ‘black boxes’. Their internal workings, involving millions of parameters calibrated on vast datasets, are typically opaque to human observers. This makes explaining how a specific inference was derived difficult, unlike traditional statistical models.

Regulations like the EU’s General Data Protection Regulation (GDPR) are introducing a “right to explanation,” allowing individuals to demand justification for decisions made via automated processing (s28). This poses a challenge for ML systems whose decision-making pathways aren’t easily articulated.

A Harvard Berkman Klein working group explored AI accountability (s29). They argue that while statistical evidence might suffice for well-defined problems, clinical practice often involves ambiguous objectives and external factors, necessitating explanation for accountability. Explanation means allowing an observer to understand how specific inputs influenced an output. They recommend AI systems provide explanations in situations where a human decision-maker would be expected to. Achieving this requires technological investment in creating separate explanation systems capable of interpreting and communicating the inner workings of complex ML algorithms, assuming this is feasible.

Managing Strategic Change and Workforce Adaptation

For ML-driven diagnosis, care management, and monitoring to gain traction, demonstrating algorithmic superiority isn’t enough. Clinicians and policymakers need evidence from experimental trials or real-world performance observations showing tangible benefits. However, ML is a constantly evolving field; algorithms improve as data availability and techniques advance, potentially requiring repeated validation efforts. This incurs costs that health systems must offset through performance gains and workforce efficiencies.

Uncertainty about ML’s impact on the workforce—within and beyond healthcare—concerns policymakers. The ‘displacement effect’ will likely hit lower-skilled occupations hardest. In health systems with sufficient workforce numbers, this could create a larger pool of workers seeking roles less prone to automation, such as those involving psychological support, emotional well-being, and care for the elderly and disabled. Coupled with ML systems, this increased supply of front-line workers could improve chronic disease management and community care for aging populations. However, governments must proactively invest in retraining displaced workers for new opportunities, including roles in data curation and ML algorithm development.

Conversely, in low- and middle-income countries (LMICs) facing severe health workforce shortages, machine learning offers a tangible opportunity to expand service coverage and accelerate progress towards universal health coverage.

Conclusions

This discussion has focused on the direct impact of artificial intelligence, machine learning, and health systems, without delving into the indirect effects of ML advancements in basic sciences, drug discovery, and other enabling technologies.

Predicting the future is inherently challenging; technology shapes its environment, which in turn creates new opportunities and constraints for the technology. While true artificial general intelligence, akin to the human brain, seems unlikely within the next 5-10 years based on current trajectory extrapolation, a more immediate prospect exists.

What is plausible, and requires planning, is the emergence of a federation of ‘narrow’ or ‘targeted’ ML systems. These systems can address core information processing challenges throughout a health system by augmenting human decision-making capabilities. This approach could establish new benchmarks for clinical and operational effectiveness and efficiency. The potential for significant health system transformation is substantial, particularly as the cost of augmenting decision-making via ML is unlikely to scale proportionally with impact—a unique advantage over other approaches. While the fixed costs of developing ML solutions (R&D, system re-tooling) are considerable, the potential for scalability provides a clear rationale for investment.

An opportunity exists to catalyze growth in artificial intelligence, machine learning, and health systems by creating high-resolution clinical datasets and establishing mechanisms for data sharing and collaborative research to validate efficacy and safety. What is currently lacking is the leadership to drive this forward. While the academic AI community actively discusses these issues, solutions require broader engagement from policymakers, citizens, patients, and clinicians. Fears of wholesale health workforce displacement by AI may be exaggerated. The more pertinent fear should be the opportunity cost of not embracing AI—of maintaining the status quo with piecemeal implementations that fail to realize the technology’s transformative potential for health systems globally.

References

1. Atun R. Transitioning health systems for multimorbidity. Lancet. 2015;386:721–2. doi: 10.1016/S0140-6736(14)62254-6. [DOI] [PubMed] [Google Scholar]
2. Kocher R, Sahni NR. Rethinking health care labor. N Engl J Med. 2011;365:1370–2. doi: 10.1056/NEJMp1109649. [DOI] [PubMed] [Google Scholar]
3. Badawi O, Brennan T, Celi LA, Feng M, Ghassemi M, Ippolito A, et al. Making big data useful for health care: a summary of the inaugural mit critical data conference. JMIR Med Inform. 2014;2:e22. doi: 10.2196/medinform.3447. [DOI] [PMC free article] [PubMed] [Google Scholar]
4. Jones SS, Heaton PS, Rudin RS, Schneider EC. Unraveling the IT productivity paradox—lessons for health care. N Engl J Med. 2012;366:2243–5. doi: 10.1056/NEJMp1204980. [DOI] [PubMed] [Google Scholar]
5. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436. doi: 10.1038/nature14539. [DOI] [PubMed] [Google Scholar]
6. Beam A, Kohane I. Big data and machine learning in health care. JAMA. 2018;319:1317–8. doi: 10.1001/jama.2017.18391. [DOI] [PubMed] [Google Scholar]
7. Marcus G. Deep learning: A critical appraisal. arXiv:1801.00631. 2018.
8. Atun R, Aydın S, Chakraborty S, Sümer S, Aran M, Gürol I, et al. Universal health coverage in Turkey: enhancement of equity. Lancet. 2013;382:65–99. doi: 10.1016/S0140-6736(13)61051-X. [DOI] [PubMed] [Google Scholar]
9. Henglin M, Stein G, Hushcha PV, Snoek J, Wiltschko AB, Cheng S. Machine learning approaches in cardiovascular imaging. Circ Cardiovasc Imaging. 2017;10:e005614. doi: 10.1161/CIRCIMAGING.117.005614. [DOI] [PMC free article] [PubMed] [Google Scholar]
10. Stanford University. Algorithm outperforms radiologists at diagnosing pneumonia [Internet]. Stanford News. 2017. Available: https://news.stanford.edu/2017/11/15/algorithm-outperforms-radiologists-diagnosing-pneumonia/. Accessed: 20 March 2018.
11. Johnson AE, Pollard TJ, Mark RG. 2017, November. Reproducibility in critical care: a mortality prediction case study. Machine Learning for Healthcare Conference 2017. JMLR W&C Track Volume 68. Available: http://proceedings.mlr.press/v68/johnson17a/johnson17a.pdf. Accessed: 20 March 2018.
12. Celi LA, Moseley E, Moses C, Ryan P, Somai M, Stone D, et al. From pharmacovigilance to clinical care optimization. Big Data. 2014;2:134–41. doi: 10.1089/big.2014.0008. [DOI] [PMC free article] [PubMed] [Google Scholar]
13. Brynjolfsson E, Mcafee AN. The business of artificial intelligence. Harv Bus Rev. 2017. Available: https://hbr.org/cover-story/2017/07/the-business-of-artificial-intelligence. Accessed: 28 September 2018.
14. Helpman E, Trajtenberg M. Diffusion of general purpose technologies. National Bureau of Economic Research. 1996. No. w5773.
15. Trajtenberg M. AI as the next GPT: a Political-Economy Perspective. National Bureau of Economic Research. 2018. No. w24245.
16. Acemoglu D, Restrepo P. Artificial intelligence, automation and work. National Bureau of Economic Research 2018. No. w24196.
17. Siddartha M. The algorithm will see you now. New Yorker. 2017;93:46–53. [Google Scholar]
18. Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, et al. CheXNet: radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv:1711.05225v3 [cs.CV].
19. Golden JA. Deep learning algorithms for detection of lymph node metastases from breast cancer: helping artificial intelligence be seen. JAMA. 2017;318:2184–6. doi: 10.1001/jama.2017.14580. [DOI] [PubMed] [Google Scholar]
20. Bychkov D, Linder N, Turkki R, Nordling S, Kovanen PE, Verrill C, et al. Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci Rep. 2018;8:3395. doi: 10.1038/s41598-018-21758-3. [DOI] [PMC free article] [PubMed] [Google Scholar]

(Note: Additional references s21-s61 mentioned in the text are detailed in the Online Supplementary Document linked in the original source material.)