Welcome to the stitch-testing app!

This app is meant to be a containerized service that runs checks on our stitching pipeline to make sure everything is running as expected. If they are not, the stitch bot in slack simply lets us know!

This app depends on the host machine having docker engine installed as all functionality is containerized. The host machine should also appropriately have ssh keys configured to access the BVC-Neurosurgery project. As far as I know there are no other dependencies other than having your .env set appropriately for your host.

To test the app

simply execute the run_tests.sh script to run a suite of tests to ensure the testing functions work properly. NOTE: this test suite only depends on the mysql server composed along with the test service. The EMU24Schema server is mocked for the sake of this test suite.

To run the app

simply execute the start_service.sh script and the server will start appropriately, running routine checks every 5 minutes and a more extensive check for file changes every 2 hours (subject to change)

Summary of scripts (in order of execution)

check_new_patients_datalake.py

• checks whether there are new patients in datalake and appropriately creates a timestamped entry for them in the stitch tracking database

check_and_populate_new_patients.py

• populates stitched folders and files for any new patients if stitched folders are available.
• If they are not available and it has been 20 minutes since patient was added to the tracking database, notifies slack

resolve_patient.py

• gets the most recent patient based on when they were added to datalake

check_and_populate_new_stitched.py

• populates stiched folders and files for specified existing patient

validate_tests.py

• validates whether test task folders are populated properly in stitched
• if not, tries to figure out what may be missing in emu db (comments, chunks, etc.)

check_file_integrity.py

• attempts to open nev/nsx files to ensure they are not corrupted
• makes sure each stitch folder contains at least an nev file
• notifies for folders/files that do not satisfy above conditions

check_stitch_population.py

• queries emu db to find unclosed tasks
• notifies slack if there are any tasks that have not been closed for at least 2 hours
• ensures all tasks in emu db with a start and stop comment have an associated folder in stitched
• if not, notifies slack
• also notifies whether the stitched chunk for missing tasks is available in emu db

NOTE: next function is run as part of a separate job that runs every 2 hours

check_file_changes.py

• finds files that have been changed (for whatever reason, replacing corrupt files, mistakes, etc.) via md5sum and marks them to be rechecked by check_file_integrity next time it runs

Extra files

there is a matlab script testing/stitch_test.m that can run a suite of test tasks at the start of a patient stay so we can ensure the stitching pipeline is working as expected early on. this script is entirely separate from the stitch-testing application and is included in the repo for convenience.