mirror of https://github.com/ollama/ollama.git
ea79003180
Computing an attention mask for a large context and max batch is expensive - over 100ms. Models like Gemma3 that have multiple types of caches and custom attention masks need to do this 4 times, so this adds approximately 500ms to startup time when using 128k context When we are reserving the worst case graph, we don't need the mask, only its shape, so we can skip this. |
||
|---|---|---|
| .. | ||
| cache.go | ||
| causal.go | ||
| causal_test.go | ||
| encoder.go | ||
| wrapper.go | ||