⚕️ Interpretable Clinical Decision Rules ⚕️️

Validating and deriving clinical-decision rules. Work-in-progress.

This is a collaborative repository intended to validate and derive clinical-decision rules. We use a unified pipeline across a variety of contributed datasets to vet previous modeling practices for clinical decision rules. Additionally, we hope to externally validate the rules under study here with data from UCSF.

Rule derivation datasets

Research paper 📄, Data download link 🔗

Datasets are all tabular (or at least have interpretable input features), reasonably large (e.g. have at least 100 positive and negative cases), and have a binary outcome. For PECARN datasets, please read and agree to the research data use agreement on the PECARN website.

Possible data sources: PECARN datasets | Kaggle datasets | MDCalc | UCI | OpenML | MIMIC | UCSF De-ID Potential specific datasets: Maybe later will expand to other high-stakes datasets (e.g. COMPAS, loan risk).

Contributing checklist

To contribute a new project (e.g. a new dataset + modeling), create a pull request following the steps below. The easiest way to do this is to copy-paste an existing project (e.g. iai_pecarn) into a new folder and then edit that one.

Helpful docs: Collaboration details | Lab writeup | Slides

• [ ] Repo set up
• [ ] Create a fork of this repo (see tutorial on forking/merging here)
• [ ] Install the repo as shown below
• [ ] Select a dataset - once you've selected, open an issue in this repo with the name of the dataset + a brief description so others don't work on the same dataset
• [ ] Assign a project_name to the new project (e.g. iai_pecarn)
• [ ] Data preprocessing
• [ ] Download the raw data into data/{project_name}/raw
  • Don't commit any very large files
• [ ] Copy the template files from rulevetting/projects/iai_pecarn to a new folder rulevetting/projects/{project_name}
  • [ ] Rewrite the functions in dataset.py for processing the new dataset (e.g. see the dataset for iai_pecarn)
  • [ ] Document any judgement calls you aren't sure about using the dataset.get_judgement_calls_dictionary function
    • See the template file for documentation of each function or the API documentation
  • Notebooks / helper functions are optional, all files should be within rulevetting/projects/{project_name}
• [ ] Data description
• [ ] Describe each feature in the processed data in a file named data_dictionary.md
• [ ] Summarize the data and the prediction task in a file named readme.md. This should include basic details of data collection (who, how, when, where), why the task is important, and how a clinical decision rule may be used in this context. Should also include your names/affiliations.
• [ ] Modeling
• [ ] Baseline model - implement baseline.py for predicting given a baseline rule (e.g. from the existing paper)
  • should override the model template in a class named Baseline
• [ ] New model - implement model_best.py for making predictions using your newly derived best model
  • also should override the model template in a class named Model
• [ ] Lab writeup (see instructions)
• [ ] Save writeup into writeup.pdf + include source files
• Should contain details on exploratory analysis, modeling, validation, comparisons with baseline, etc.
• [ ] Submitting
• [ ] Ensure that all tests pass by running pytest --project {project_name} from the repo directory
• [ ] Open a pull request and it will be reviewed / merged
• [ ] Reviewing submissions
• [ ] Each pull request will be reviewed by others before being merged

Installation

Note: requires python 3.7 and pytest (for running the automated tests). It is best practice to create a venv or pipenv for this project.

python -m venv rule-env
source rule-env/bin/activate

Then, clone the repo and install the package and its dependencies.

git clone https://github.com/Yu-Group/rule-vetting
cd rule-vetting
pip install -e .

Now run the automatic tests to ensure everything works.

pytest --project iai_pecarn

To use with jupyter, might have to add this venv as a jupyter kernel.

python -m ipykernel install --user --name=rule-env

Clinical Trial Datasets

Research paper 📄, Data download link 🔗

Reference

"""
.. include:: ../readme.md
"""
import os
from os.path import join as oj

MRULES_PATH = os.path.dirname(os.path.abspath(__file__))
REPO_PATH = os.path.dirname(MRULES_PATH)
DATA_PATH = oj(REPO_PATH, 'data')
PROJECTS_PATH = oj(MRULES_PATH, 'projects')
AUTOGLUON_CACHE_PATH = oj(DATA_PATH, 'autogluon_cache')

Sub-modules

rulevetting.api

rulevetting.projects

rulevetting.templates