Skip to content

Honor unk_token in byte-level BPE - #2736

Open
galleonli wants to merge 1 commit into
huggingface:mainfrom
galleonli:fix/byte-level-bpe-unknowns
Open

galleonli wants to merge 1 commit into
huggingface:mainfrom
galleonli:fix/byte-level-bpe-unknowns

Honor unk_token in byte-level BPE

149411c
Select commit
Loading
Failed to load commit list.