Explore better models & architecture

[tenzinyonten]

Description

TiSpell (RoBERTa + copy-gate) has architectural limitations when handling multi-syllable edits and complex Tibetan glyph stack corruptions. Explore, fine-tune, and benchmark alternative sequence-to-sequence and LLM architectures against TiSpell on an audited CSC evaluation set to determine the most effective production pipeline.

Subtask

• Document candidate architectures, focusing on byte-level seq2seq (google/byt5-base), subword seq2seq (mT5), character-level models, and prompted/fine-tuned LLMs (Gemini).
• ByT5 Fine-Tuning: Fine-tune google/byt5-base on v11_train_ready.csv.