2 Commits

Author	SHA1	Message	Date
Jeffrey Morgan	9e125d884c	model: treat 'user defined' tokens as special tokens (#11077)	2025-06-16 16:03:16 -07:00
Michael Yang	333e360422	model: handle multiple eos tokens (#10577) ... * get eos_token_id from generation_config.json * refactor * include both ids and strings in trace * comments * remove special case for gemma3 special vocab (#10743)	2025-05-16 13:40:23 -07:00