This repository contains the framework for reproducing the key learning experiments from our study on dynamic prompting, an in-context learning method built for outcome prediction with EHR notes. Initially developed for our manuscript submission to AAPM 2024, and received the Best in Physics award. Enhancements and updates will be tracked through specific commit checkpoints, and as the codebase improves, results may exhibit slight variations from those in the original study.
This repository provides a structured framework for running machine learning experiments on synthetic and real-world clinical notes. It consists of two main folders:
This folder contains high-fidelity synthetic data designed to closely replicate the format of UCSF Glioma and Breast Cancer EHR notes. It includes:
- Full clinical notes with their corresponding labels.
- Summarized versions of the notes, generated using GPT-4.
- Cohort coverage: 15 patients each for Glioma and Breast Cancer datasets.
This folder contains the core pipeline for executing various learning setups, including:
- Zero-shot learning, few-shot learning with dynamic prompting, and summarization-enhanced approaches.
- The RESULTS.ipynb notebook, showcasing results from running UCSF Glioma and Breast cancer data on the Llama 3.0 8B model and Llama 3.0 70B model.
- Additional documentation in the README.md file within the pipeline folder, detailing implementation and usage.
We appreciate contributions that help enhance the learning experiments and improve this framework. If you'd like to contribute, please follow these steps:
- Fork the repository to your GitHub account.
- Create a new branch for your changes:
git checkout -b <your_branch_name>
- Make your modifications and commit them with a clear message:
git commit -m "Your commit message" - Push your changes to your forked repository:
git push origin <your_branch_name>
- Submit a pull request (PR) to the main repository for review.
We'd like to thank the following contributors for their valuable work on this project:
And a special thanks to all other contributors who have improved the project over time!
For any scientific inquiries about this repository, please contact hui.lin@ucsf.edu