Real-World Evidence & Systems Integration

The Challenge

While AI holds tremendous promise for oncology, two critical challenges limit its real-world impact. First, most AI models are developed and validated using carefully curated research datasets that often fail to represent the complexity, variability, and messiness of real-world clinical data. This leads to a significant gap between published performance metrics and actual clinical utility.

Second, even the most accurate AI tools often fail to achieve clinical adoption and impact due to poor integration with existing healthcare systems and workflows. Clinicians face alert fatigue, disrupted workflows, technical barriers, lack of transparency, and misalignment with their decision-making processes—all contributing to limited real-world utilization and impact of AI technologies in cancer care.

Our Approach

We aim to address these challenges through a dual-focused research agenda that combines real-world evidence generation with healthcare systems integration. Our multifaceted approach includes:

Developing AI methods specifically designed for the complexities of real-world data, including missing values, inconsistent measurements, and diverse documentation practices
Creating frameworks for rigorous assessment of AI performance in real-world settings across diverse patient populations and care environments
Designing AI systems that adaptively integrate into clinical workflows, providing the right information at the right time in the right format to support clinical decision-making
Building implementation science approaches that identify and address barriers to AI adoption in oncology practice
Establishing methodologies for continuous monitoring and improvement of AI systems after deployment in clinical settings

Real-World Performance Assessment

Methods to rigorously evaluate how AI systems perform in actual clinical settings with diverse patient populations and practice patterns.

Workflow Integration

Approaches to seamlessly embed AI tools into clinical workflows, minimizing disruption while maximizing utility and adoption.

Implementation Science

Research on the organizational, technical, and human factors that influence successful adoption of AI systems in oncology practice.

Learning Healthcare Systems

Creating frameworks where AI systems continuously improve based on real-world usage data and outcomes, forming a virtuous cycle of enhancement.

Potential Research Projects

Adaptive Clinical Decision Support

We could develop AI-powered clinical decision support systems that adapt their behavior based on context, user preferences, and previous interactions. Unlike traditional rule-based alerts, these systems would learn when, how, and what information to present to maximize clinical utility while minimizing disruption. For example, the system might adjust the timing, format, and content of recommendations based on the specific clinician's workflow patterns, the patient complexity, and the clinical context, potentially increasing adoption and impact.

Real-World Validation Framework

This project would create a comprehensive methodology for assessing AI performance in real-world oncology settings. It would include approaches for identifying distribution shifts between development and deployment environments, methods for continual performance monitoring across diverse subpopulations, techniques for detecting and addressing performance degradation over time, and standards for transparent reporting of real-world metrics that matter to clinicians and patients.

Electronic Health Record Integration Platform

We aim to develop a modular platform that simplifies the integration of AI tools with diverse electronic health record (EHR) systems. This platform would address key technical challenges including standardized data extraction and normalization, secure API connections, bidirectional information flow, and audit capabilities. By creating a consistent integration framework, we could reduce the implementation burden for healthcare systems and accelerate the translation of promising AI tools into clinical practice.

Implementation Barriers Observatory

This project would systematically identify, categorize, and address barriers to successful AI implementation in oncology practice. Through mixed-methods research combining qualitative studies, workflow analysis, and quantitative metrics, we would develop a comprehensive understanding of technical, organizational, and human factors affecting adoption. The resulting knowledge base would inform both AI system design and implementation strategies to maximize the likelihood of successful clinical integration and impact.

Technical Innovations

Our real-world evidence and systems integration research would leverage several cutting-edge technical approaches:

Robust learning algorithms: Methods designed to maintain performance despite the noise, missingness, and variability in real-world clinical data
Transfer learning: Techniques to adapt models across different healthcare settings and populations while preserving performance
Continuous learning systems: Approaches for models to safely update and improve based on new real-world data after deployment
Explainable AI: Methods that provide transparent rationales for recommendations in clinically meaningful terms
Human-computer interaction: Advanced interfaces that present AI insights in ways that complement human clinical reasoning
Process mining: Techniques to understand and optimize clinical workflows for effective AI integration

Data & Evidence Generation

Our research would address multiple dimensions of real-world data and evidence generation:

EHR Data Utilization

Methods to effectively extract, normalize, and analyze the rich but messy data contained in electronic health records for oncology insights.

Claims Data Analytics

Approaches to leverage administrative data for understanding treatment patterns, outcomes, and disparities across broad populations.

Multi-Modal Data Fusion

Techniques to combine structured clinical data with unstructured notes, images, genomics, and patient-reported data for comprehensive insights.

Integration Contexts

Our systems integration research would address diverse healthcare contexts:

Specialty Oncology Care

Integrating AI tools into cancer centers and oncology practices to support complex decision-making and multidisciplinary care coordination.

Primary Care Settings

Developing AI approaches to support cancer screening, early detection, and survivorship care in general practice environments.

Multidisciplinary Tumor Boards

Creating collaborative AI tools that enhance group decision-making processes for complex cancer cases across specialties.

Clinical Trial Matching

Building systems that seamlessly integrate trial eligibility screening into routine clinical workflows to increase participation rates.

Implementation Considerations

Successful healthcare systems integration requires addressing several key factors:

Clinical workflow analysis: Understanding existing processes to minimize disruption and maximize value from AI integration
Stakeholder engagement: Involving clinicians, administrators, patients, and IT personnel in design and implementation processes
Technical infrastructure: Addressing computing requirements, data pipelines, security, and interoperability challenges
Change management: Developing effective training, communication, and support strategies for successful adoption
Regulatory considerations: Navigating FDA requirements, liability concerns, and other regulatory challenges
Business model alignment: Ensuring AI implementation aligns with financial incentives and organizational priorities

Future Directions

As our research progresses, we plan to explore several exciting directions:

Developing standardized benchmarks for assessing real-world AI performance in oncology across diverse clinical settings
Creating federated learning approaches that enable AI models to continuously improve from distributed clinical data while preserving privacy
Building collaborative AI systems designed to enhance human-AI teamwork rather than simply automating clinical tasks
Exploring novel incentive models and reimbursement approaches to support sustainable implementation of effective AI tools
Establishing comprehensive frameworks for measuring the clinical, operational, and economic impact of AI interventions in cancer care

Collaborations and Partnerships

We would be interested in exploring partnerships with:

Healthcare Systems

To conduct real-world validation studies and explore effective implementation approaches

EHR Vendors

To develop standards and interfaces for seamless integration of AI tools with clinical systems

Implementation Scientists

To develop and test strategies for effective adoption of AI tools in clinical practice

AI for Real-World Evidence & Healthcare Systems Integration