Skip to content

feat: add per-toolset model routing via model field on toolsets#2015

Open
dgageot wants to merge 1 commit intodocker:mainfrom
dgageot:board/fix-this-per-tool-model-routing-gordon-u-d38bede0
Open

feat: add per-toolset model routing via model field on toolsets#2015
dgageot wants to merge 1 commit intodocker:mainfrom
dgageot:board/fix-this-per-tool-model-routing-gordon-u-d38bede0

Conversation

@dgageot
Copy link
Member

@dgageot dgageot commented Mar 9, 2026

Add a 'model' field to toolset configuration that automatically routes the LLM turn processing tool results through a specified model. This enables platform-level cost optimization: cheaper/faster models handle simple tool results (file reads, knowledge-base lookups) while the agent's primary model handles complex reasoning.

Config & schema:

  • Add Model field to Toolset struct in pkg/config/latest/types.go
  • Add model property to Toolset in agent-schema.json

Tool plumbing:

  • Add ModelOverride field to Tool struct (pkg/tools/tools.go)
  • Add WithModelOverride wrapper in pkg/teamloader/model_override.go
  • Wire wrapper into getToolsForAgent in pkg/teamloader/teamloader.go

Runtime:

  • Extract reusable resolveModelRef from SetAgentModel in model_switcher.go with proper alloy config rejection
  • Add resolveToolCallModelOverride helper (pkg/runtime/tool_model_override.go)
  • In RunStream loop: after tool calls, record override; on next iteration, temporarily switch model for one turn then reset
  • Clear override on agent switch to prevent cross-agent leakage

Tests:

  • Unit tests for WithModelOverride wrapper
  • Unit tests for resolveToolCallModelOverride
  • Unit tests for resolveModelRef (alloy rejection, nil config, invalid format)
  • Example config: per_tool_model_routing.yaml

Assisted-By: docker-agent

@dgageot dgageot requested a review from a team as a code owner March 9, 2026 18:30
Copy link

@docker-agent docker-agent bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

Assessment: 🟢 APPROVE

This PR adds per-toolset model routing to enable cost optimization by routing simple tool results through cheaper models. The implementation is clean and well-structured:

✅ Strengths:

  • Proper separation of concerns: config schema → tool plumbing → runtime integration
  • Correct control flow with early returns preventing fall-through bugs
  • Comprehensive test coverage for new functions (resolveModelRef, WithModelOverride, resolveToolCallModelOverride)
  • Good validation: resolveModelRef properly rejects alloy configs, isInlineAlloySpec validates format
  • Clear override lifecycle management (set on tool calls, clear on agent switch)
  • Example config demonstrating the feature

🔍 Code Quality:

  • The refactoring that extracted resolveModelRef maintains the original behavior correctly
  • Alloy spec handling uses mutually exclusive code paths with proper returns
  • Model override wrapper pattern is clean and composable
  • Error messages are descriptive

No bugs found in the changed code. The feature is ready to merge.


Review completed by cagent

rumpl
rumpl previously approved these changes Mar 10, 2026
Add a 'model' field to toolset configuration that automatically routes
the LLM turn processing tool results through a specified model. This
enables platform-level cost optimization: cheaper/faster models handle
simple tool results (file reads, knowledge-base lookups) while the
agent's primary model handles complex reasoning.

Config & schema:
- Add Model field to Toolset struct in pkg/config/latest/types.go
- Add model property to Toolset in agent-schema.json

Tool plumbing:
- Add ModelOverride field to Tool struct (pkg/tools/tools.go)
- Add WithModelOverride wrapper in pkg/teamloader/model_override.go
- Wire wrapper into getToolsForAgent in pkg/teamloader/teamloader.go

Runtime:
- Extract reusable resolveModelRef from SetAgentModel in model_switcher.go
  with proper alloy config rejection
- Add resolveToolCallModelOverride helper (pkg/runtime/tool_model_override.go)
- In RunStream loop: after tool calls, record override; on next iteration,
  temporarily switch model for one turn then reset
- Clear override on agent switch to prevent cross-agent leakage

Tests:
- Unit tests for WithModelOverride wrapper
- Unit tests for resolveToolCallModelOverride
- Unit tests for resolveModelRef (alloy rejection, nil config, invalid format)
- Example config: per_tool_model_routing.yaml

Assisted-By: docker-agent
@dgageot dgageot force-pushed the board/fix-this-per-tool-model-routing-gordon-u-d38bede0 branch from faad0e7 to fe20490 Compare March 10, 2026 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants