add OASM model from Hadidi et al. 2025#355
Conversation
|
Retriggering orchestrator |
|
@mschrimpf Currently scoring is still coming online. I can add a cursor generated |
…eproducing Figure 1 on Pereira2018 with both OASM sigma sweep and GPT2-XL, including per-subject evaluation. Replaced with N x N identity matrix with per-block Guassian smoothing (sigma searched over 48 values as per paper). Add tests Co-written by Claude Code (Opus 4.6 High effort) and requiring multiple iterations especially during notebook validation.
|
Attempt at reproducing OASM and the results shown in the paper. Used inspiration from #356 and #361 to match evaluation approach (ridge + story-level groups). Used GPT2-XL to validate analysis. Unclear why the differences in GPT2-XL for contiguous
The colored dots/bar are from paper's source data. Stars represent re-implementation. NOTE: It is probably best not to have all 48 models as public. Will choose sigma0.9 after tests pass on all of them |
mike-ferguson
added
the
submission_prepared
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Thanks for doing this! |
4 participants
Cursor-aided implementation based on the paper.
Preliminary results from local run:
0.34on Pereira2018-linear