YaCy '#[clientname]#': LLM Selection

#%env/templates/metas.template%# #%env/templates/header.template%# #%env/templates/submenuAI.template%#

LLM Selection

Here you can pick models from a LLM model service to select them as production model. In the "Production Models Matrix" you can then assign each selected model a function inside YaCy

Install your local LLM service! You need either a local ollama or LM Studio instance running on your local host or inside the intranet.

Service Selection

service: This makes a preset to the Hoststub value
hoststub: you can probably leave this to the default value
api_key: (not required for Ollama or LMStudio)
max_tokens: This is the default max_tokens applied to a model when you deploy it; you can change it per model afterwards in the Production Models Matrix below.
max_tokens caps the number of generated tokens (sent as the OpenAI max_tokens, i.e. Ollama num_predict). It does not enlarge the model's context window: Ollama defaults num_ctx to 4096 regardless of this value, and its OpenAI-compatible endpoint (/v1/chat/completions) cannot set num_ctx per request. To actually use a large context in Ollama, raise the context length once via the environment variable OLLAMA_CONTEXT_LENGTH, a Modelfile PARAMETER num_ctx <n>, or the Context Length slider in the Ollama app settings.

Production Models Matrix #{productionmodels}# #{/productionmodels}#

service	model	hoststub	api_key	max_tokens	search-answers This model creates answers for search requests	chat This model is used in the chat interface and as default for the RAG proxy	translation This model can be used to make translations of the web UI	classification This model is used to classify prompts to find out what they demand	search-query This model produces search queries to YaCy search from prompts in RAG or chat	qa-pairs This model can be used to produce query-answer pairs which enhance search from chat prompts	tldr-shortener This model is used to make summaries from web content	log-report This model evaluates YaCy runtime logs and creates self-enhancement reports	thinking we detect thinking only to be able to suppress thinking. thinking is not used in YaCy	tooling tooling is required for agentic abilities.	vision this enables image recognition in the chat	format this is required for classification	Actions
#[service]#	#[model]#	#[hoststub]#	#[api_key]#	#[max_tokens]#									#[thinking]#	#[tooling]#	#[vision]#	#[format]#

#%env/templates/footer.template%#