diff --git a/docs/docs/configuration/genai/config.md b/docs/docs/configuration/genai/config.md index 738eb5db3f..d1da92c05f 100644 --- a/docs/docs/configuration/genai/config.md +++ b/docs/docs/configuration/genai/config.md @@ -78,7 +78,7 @@ All llama.cpp native options can be passed through `provider_options`, including - Set **Provider** to `llamacpp` - Set **Base URL** to your llama.cpp server address (e.g., `http://localhost:8080`) - Set **Model** to the name of your model - - Under **Provider Options**, set `context_size` to tell Frigate your context size so it can send the appropriate amount of information + - Optionally, under **Provider Options**, set `context_size` to override the context size Frigate detects from the server @@ -89,12 +89,14 @@ genai: base_url: http://localhost:8080 model: your-model-name provider_options: - context_size: 16000 # Tell Frigate your context size so it can send the appropriate amount of information. + context_size: 16000 # Optional, overrides the context size reported by the server. ``` +Frigate queries the llama.cpp server for the model's context size at startup and logs it along with the other detected capabilities. If `context_size` is set in `provider_options`, that value is always used instead, even when the server reports its own. + ### Ollama [Ollama](https://ollama.com/) allows you to self-host large language models and keep everything running locally. It is highly recommended to host this server on a machine with an Nvidia graphics card, or on a Apple silicon Mac for best performance.