Benjamin Trent 9dc8aea1cb [ML] adds new mpnet tokenization for nlp models (#82234) ... This commit adds support for MPNet based models. MPNet models differ from BERT style models in that: - Special tokens are different - Input to the model doesn't require token positions. To configure an MPNet tokenizer for your pytorch MPNet based model: ``` "tokenization": { "mpnet": {...} } ``` The options provided to `mpnet` are the same as the previously supported `bert` configuration.		2022-01-05 12:56:47 -05:00
..
apis	[ML] adds new mpnet tokenization for nlp models (#82234)	2022-01-05 12:56:47 -05:00