Skip to content

[RAG] embedding_model should support custom provider #1701

@ConfusedOnCOFFEE

Description

@ConfusedOnCOFFEE

What you'd like to see

I would like to have embedding_model to also use the custom models, which are defined in the same agent file.

Why you'd like to see it

  • Consistency about provider usage
  • Self hosting option for RAG as well

Workarounds?

Might be also a fix if the embedding_model is checked against all provider/model options

Additional context

-> Supporting OLLAMA_HOST and OLLAMA_KEY might also help if the normal definition 'ollama/llama3' would respect overwrite models.

Defining:

providers:
  ollama:
    base_url: myurl:11434

won't be respected.

Defining:

models:
  local-ollama:
    provider: ollama
    models: llama3

....
    embedding_models: local-ollama

also does not work.

Test example:

providers:
  confollama:
    base_url: http://#OLLAMA_IP:11434/v1
  ollama:
    base_url: http://$OLLAMA_IP:11434/v1
models:
  nomic:
    provider: confollama
    model: nomic-embed-text:latest
    max_tokens: 8192
  mistral:
    provider: confollama
    model: ministral-3:3b
    max_tokens: 8192

rag:
  my_knowledge_base:
    docs:
      - ./documents
    strategies:
      - type: chunked-embeddings
        embedding_model: ollama/qwen3-embeddings
        database: ./rag/chunked_embeddings.db
        vector_dimensions: 1536

agents:
  root:
    model: mistral
    instruction: |
      You are an assistant with access to an internal knowledge base.
      Use the knowledge base to gather context before answering user questions
    rag: [my_knowledge_base]

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/providersFor features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)kind/enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions