ollama/model
Devon Rifkin 242df70a75 parsers: fix `&`s in qwen3coder parameter values
In <https://github.com/ollama/ollama/issues/12357> we that the model
will output tool calls such as

```
<function=shell>
<parameter=command>
pwd && ls -la
</parameter>
</function>
```

We parse this using the approach of transforming into valid xml and then
using an xml parser. While we do transform the function and parameter
names, we weren't escaping the parameter values (which in this example
are invalid since `pwd && ls -la` contains unescaped ampersands).

This has been fixed by first transforming the tags in the same way, and
then walking the transformed string and escaping the text in between the
tags. This also fixes a case where `<` in the middle of a parameter
value would cause an xml parse failure.

Fixes: #12357
2025-09-20 12:11:38 -07:00
..
imageproc imageproc mllama refactor (#7537) 2024-12-14 19:50:15 -08:00
input batch: use tensors for outputs (#12185) 2025-09-15 14:33:06 -07:00
models gemma: fix rope scaling for qat models (#12348) 2025-09-19 15:04:40 -07:00
parsers parsers: fix `&`s in qwen3coder parameter values 2025-09-20 12:11:38 -07:00
renderers address comments 2025-09-15 11:46:25 -07:00
testdata gemma2 impl 2025-03-11 14:35:08 -07:00
bytepairencoding.go embedding gemma model (#12181) 2025-09-04 09:09:07 -07:00
bytepairencoding_test.go model: add bpe roundtripping tests 2025-08-19 22:05:48 -07:00
model.go fix: model load for unsupported embedding models (#12311) 2025-09-18 16:11:08 -07:00
model_test.go fix: model load for unsupported embedding models (#12311) 2025-09-18 16:11:08 -07:00
sentencepiece.go model: implement bert in ollama engine (#9080) 2025-09-15 15:35:59 -07:00
sentencepiece_test.go model: implement bert in ollama engine (#9080) 2025-09-15 15:35:59 -07:00
textprocessor.go model: handle multiple eos tokens (#10577) 2025-05-16 13:40:23 -07:00
vocabulary.go embedding gemma model (#12181) 2025-09-04 09:09:07 -07:00
vocabulary_test.go model: treat 'user defined' tokens as special tokens (#11077) 2025-06-16 16:03:16 -07:00
wordpiece.go model: implement bert in ollama engine (#9080) 2025-09-15 15:35:59 -07:00
wordpiece_test.go model: implement bert in ollama engine (#9080) 2025-09-15 15:35:59 -07:00