Add `DynamicToolset` support in Temporal #3682

dsfaccini · 2025-12-09T13:39:22Z

Closes #3390

adds support for @agent.toolset decorator

docs/durable_execution/temporal.md

docs/toolsets.md

pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_toolset.py

pydantic_ai_slim/pydantic_ai/toolsets/_dynamic.py

pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_dynamic_toolset.py

DouweM · 2025-12-09T22:33:25Z

tests/test_temporal.py

+        assert output == 'Dynamic result received'
+
+
+# Test with explicit id parameter


I'd rather test the new id arg where we test the @agent.toolset decorator, since this part is not Temporal-specific

tests/test_temporal.py

DouweM · 2025-12-09T22:34:05Z

tests/test_temporal.py

+        pytest.fail('TemporalDynamicToolset not found in toolsets')
+
+
+async def test_dynamic_toolset_get_tools_outside_workflow():


We should test this by running the same agent we created earlier outside of a workflow, and checking that we still get the correct result

DouweM · 2025-12-09T22:34:49Z

tests/test_temporal.py

+        pytest.fail('TemporalDynamicToolset not found')
+
+
+async def test_dynamic_toolset_call_tool_outside_workflow():


DouweM · 2025-12-09T22:35:17Z

tests/test_temporal.py

+        pytest.fail('TemporalDynamicToolset not found')
+
+
+async def test_dynamic_toolset_tool_not_found_in_activity():


I'm fine with pragma: no cover for this UserError like we have here:

pydantic-ai/pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_function_toolset.py

Lines 41 to 47 in f42e523

try:

tool = (await toolset.get_tools(ctx))[name]

except KeyError as e: # pragma: no cover

raise UserError(

f'Tool {name!r} not found in toolset {self.id!r}. '

'Removing or renaming tools during an agent run is not supported with Temporal.'

) from e

Core changes: - DynamicToolset: convert to plain class with custom __init__, add copy() - TemporalDynamicToolset: use _call_tool_in_activity shared method - TemporalFunctionToolset: use _call_tool_in_activity shared method - temporalize_toolset: move DynamicToolset import to top Test changes: - Add test_fastmcp_dynamic_toolset_in_workflow for MCP lifecycle in DynamicToolset - Add test_dynamic_toolset_id and test_agent_toolset_decorator_id - Update test_visit_and_replace for DynamicToolset plain class 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>

DouweM · 2025-12-12T20:24:15Z

docs/durable_execution/temporal.md


 When `TemporalAgent` dynamically creates activities for the wrapped agent's model requests and toolsets (specifically those that implement their own tool listing and calling, i.e. [`FunctionToolset`][pydantic_ai.toolsets.FunctionToolset] and [`MCPServer`][pydantic_ai.mcp.MCPServer]), their names are derived from the agent's [`name`][pydantic_ai.agent.AbstractAgent.name] and the toolsets' [`id`s][pydantic_ai.toolsets.AbstractToolset.id]. These fields are normally optional, but are required to be set when using Temporal. They should not be changed once the durable agent has been deployed to production as this would break active workflows.

+For dynamic toolsets created with the [`@agent.toolset`][pydantic_ai.Agent.toolset] decorator, the `id` parameter can be set explicitly or it will default to the function name:


Please link to the dynamic toolset doc

I think it's actually better to NOT default to the function name and require id to be set explicitly when Temporal is used, as the ID is used in activity names that need to be stable (i.e. can't change from one release to the next). So it's better for it to be separate field with a clear docstring that the user won't accidentally rename when they're doing a code refactor

DouweM · 2025-12-12T20:24:23Z

docs/durable_execution/temporal.md

+class MyDeps:
+    ...
+
+@agent.toolset(id='my_dynamic_tools')


I don't think we need the example

DouweM · 2025-12-12T20:24:40Z

docs/durable_execution/temporal.md

+    ...
+```
+
+Note that with Temporal, `per_run_step=False` is not respected, as the toolset always needs to be created on-the-fly in the activity.


This can be on the same single line about dynamic toolsets as the stuff above

DouweM · 2025-12-12T20:28:21Z

docs/toolsets.md


 By default, the function will be called again ahead of each agent run step. If you are using the decorator, you can optionally provide a `per_run_step=False` argument to indicate that the toolset only needs to be built once for the entire run.

+When using [Temporal durable execution](./durable_execution/temporal.md), the decorator also accepts an `id` parameter to uniquely identify the toolset. If not provided, the function name is used as the ID.


This makes it seem like it only accepts the parameter when Temporal is used, while of course it always accepts the parameter :)

If we don't use the function name by default as I suggested above, we may not need this line here at all, as this won't be relevant unless they use Temporal, and if they do they'll get an error or read it in the docs

removed the wording about the id

DouweM · 2025-12-12T20:33:02Z

pydantic_ai_slim/pydantic_ai/toolsets/_dynamic.py

+    per_run_step: bool
+    _id: str | None
+    _toolset: AbstractToolset[AgentDepsT] | None
+    _run_step: int | None


I wonder if we could solve this "reuse a dynamic toolset across multiple agent runs" issue not by copying it for each run (which is tricky as we see here), but by storing the _toolset and _run_step on a dict keyed by ctx.run_id.

That way, we can keep using the same toolset instance, but state from different runs wouldn't interfere. I think that's worth trying in this PR, as an alternative to the new copy method

DouweM · 2025-12-12T20:33:50Z

tests/test_temporal.py

+# --- DynamicToolset / @agent.toolset tests ---
+
+
+def get_dynamic_weather(location: str) -> str:


We don't need this at the top level; we can register it to the FunctionToolset inline inside my_dynamic_toolset using @toolset.tool

DouweM · 2025-12-12T20:34:35Z

tests/test_temporal.py

+# --- MCP-based DynamicToolset test ---
+# Tests that @agent.toolset with an MCP toolset works with Temporal workflows.
+# Uses FastMCPToolset (HTTP-based) rather than MCPServerStdio (subprocess-based) because
+# MCPServerStdio has issues when created dynamically inside Temporal activities.


Let's link to the issue

DouweM · 2025-12-12T20:35:02Z

tests/test_temporal.py

+    """Dynamic toolset that returns an MCP toolset.
+
+    This tests MCP lifecycle management (context manager enter/exit) within DynamicToolset + Temporal.
+    Uses per_run_step=False so the toolset persists across run steps within an activity.


Does this actually work?

unsure what this was related to

Well we say "Uses per_run_step=False so the toolset persists across run steps within an activity.", but I'm not sure that actually works, as I think we regenerate the dynamic toolset each time, and we explicitly document that we don't respect per_run_step=False. And "run steps within an activity" doesn't really make sense

DouweM · 2025-12-12T20:36:21Z

tests/test_temporal.py

+
+@dynamic_toolset_agent.toolset
+def my_dynamic_toolset(ctx: RunContext[None]) -> FunctionToolset[None]:
+    return FunctionToolset(tools=[get_dynamic_weather], id='dynamic_weather')


Let's use a value from deps inside the function (when we inline it), since the point of DynamicToolset is to build it dynamically. We can do that in the MCP server example as well, if that seems more appropriate

tests/test_toolsets.py

DouweM · 2025-12-15T20:36:00Z

pydantic_ai_slim/pydantic_ai/durable_exec/temporal/_dynamic_toolset.py

+    def temporal_activities(self) -> list[Callable[..., Any]]:
+        return [self.get_tools_activity, self.call_tool_activity]
+
+    async def __aenter__(self) -> Self:


This behavior in aenter/aexit may make sense on TemporalWrapperToolset. Then we can also remove the overrides from TemporalMCPServer.

DouweM · 2025-12-15T20:38:51Z

pydantic_ai_slim/pydantic_ai/agent/__init__.py

        toolset = CombinedToolset(toolsets)

        # Copy the dynamic toolsets to ensure each run has its own instances
        def copy_dynamic_toolsets(toolset: AbstractToolset[AgentDepsT]) -> AbstractToolset[AgentDepsT]:


Now that we use run_id in DynamicToolset, I don't think we need to do this anymore, nor do we need the copy method

DouweM · 2025-12-15T20:40:13Z

pydantic_ai_slim/pydantic_ai/toolsets/_dynamic.py



-@dataclass
+class ToolsetRunStep(TypedDict, Generic[AgentDepsT]):


Could this be a dataclass?

Can we come up with a more clear name? Something with "state" in it

DouweM · 2025-12-15T20:40:50Z

pydantic_ai_slim/pydantic_ai/toolsets/_dynamic.py

-    _run_step: int | None = None
+    per_run_step: bool
+    _id: str | None
+    _toolset_runstep: dict[str, ToolsetRunStep[AgentDepsT]]


Maybe _run_state?

DouweM · 2025-12-15T20:41:50Z

pydantic_ai_slim/pydantic_ai/toolsets/_dynamic.py

+
+    def __eq__(self, other: object) -> bool:
+        if not isinstance(other, DynamicToolset):
+            return False


It reads a little easier to ad isinstance(other, DynamicToolset) and to the next lines

DouweM · 2025-12-15T20:45:32Z

pydantic_ai_slim/pydantic_ai/toolsets/_dynamic.py

        self, visitor: Callable[[AbstractToolset[AgentDepsT]], AbstractToolset[AgentDepsT]]
    ) -> AbstractToolset[AgentDepsT]:
-        if self._toolset is None:
+        wrapped_items = [(run_id, rs) for run_id, rs in self._toolset_runstep.items() if rs['toolset'] is not None]


Could this be a dict instead of a list of tuples?

DouweM · 2025-12-15T20:46:34Z

tests/test_temporal.py

+
+# --- MCP-based DynamicToolset test ---
+# Tests that @agent.toolset with an MCP toolset works with Temporal workflows.
+# See https://github.com/pydantic/pydantic-ai/issues/3390


This is the issue we're fixing, I meant let's fix to the issue that's called out in the next lines that we haven't fixed yet!

DouweM · 2025-12-15T20:47:34Z

tests/test_temporal.py

+    """Dynamic toolset that returns an MCP toolset.
+
+    This tests MCP lifecycle management (context manager enter/exit) within DynamicToolset + Temporal.
+    Uses per_run_step=False so the toolset persists across run steps within an activity.


Well we say "Uses per_run_step=False so the toolset persists across run steps within an activity.", but I'm not sure that actually works, as I think we regenerate the dynamic toolset each time, and we explicitly document that we don't respect per_run_step=False. And "run steps within an activity" doesn't really make sense

DouweM · 2025-12-15T20:47:44Z

tests/test_temporal.py

+    This tests MCP lifecycle management (context manager enter/exit) within DynamicToolset + Temporal.
+    Uses per_run_step=False so the toolset persists across run steps within an activity.
+    """
+    return FastMCPToolset('https://mcp.deepwiki.com/mcp', id='dynamic_deepwiki')


Can we use our MCPServerStreamableHTTP instead? Or does that also have the issue of MCPServerStdio? In that case, the comment above is incorrect, because it's not just MCPServerStdio that has issues, but all MCPServers

DouweM · 2025-12-15T20:52:01Z

tests/test_toolsets.py

+    # Different instances with different functions
+    assert toolset1 != toolset2
+
+    # Compare with non-DynamicToolset


This doesn't seem really worth testing

dsfaccini added 4 commits December 9, 2025 08:35

add support for toolset decorator in temporal

eefa9bf

Merge branch 'main' into add-support-for-toolset-decorator-in-temporal

130be84

Merge branch 'main' into add-support-for-toolset-decorator-in-temporal

1fc6afc

fix tests

cb1e005

DouweM requested changes Dec 9, 2025

View reviewed changes

DouweM self-assigned this Dec 9, 2025

DouweM added the awaiting author revision label Dec 9, 2025

small review changes

548c2d5

dsfaccini mentioned this pull request Dec 10, 2025

Parallel MCP servers: RuntimeError: Attempted to exit cancel scope in a different task than it was entered in #2818

Open

2 tasks

dsfaccini and others added 3 commits December 10, 2025 17:25

fix example

e0e183d

coverage

a3580c8

DouweM requested changes Dec 12, 2025

View reviewed changes

dsfaccini added 4 commits December 13, 2025 12:44

apply review fixes

c2d05ce

coverage

610a646

restor snapshot

78320ea

coverage

44e4d20

DouweM requested changes Dec 15, 2025

View reviewed changes

		assert output == 'Dynamic result received'


		# Test with explicit id parameter

		pytest.fail('TemporalDynamicToolset not found in toolsets')


		async def test_dynamic_toolset_get_tools_outside_workflow():

		pytest.fail('TemporalDynamicToolset not found')


		async def test_dynamic_toolset_call_tool_outside_workflow():

		pytest.fail('TemporalDynamicToolset not found')


		async def test_dynamic_toolset_tool_not_found_in_activity():

	try:
	tool = (await toolset.get_tools(ctx))[name]
	except KeyError as e: # pragma: no cover
	raise UserError(
	f'Tool {name!r} not found in toolset {self.id!r}. '
	'Removing or renaming tools during an agent run is not supported with Temporal.'
	) from e


		When `TemporalAgent` dynamically creates activities for the wrapped agent's model requests and toolsets (specifically those that implement their own tool listing and calling, i.e. [`FunctionToolset`][pydantic_ai.toolsets.FunctionToolset] and [`MCPServer`][pydantic_ai.mcp.MCPServer]), their names are derived from the agent's [`name`][pydantic_ai.agent.AbstractAgent.name] and the toolsets' [`id`s][pydantic_ai.toolsets.AbstractToolset.id]. These fields are normally optional, but are required to be set when using Temporal. They should not be changed once the durable agent has been deployed to production as this would break active workflows.

		For dynamic toolsets created with the [`@agent.toolset`][pydantic_ai.Agent.toolset] decorator, the `id` parameter can be set explicitly or it will default to the function name:


		By default, the function will be called again ahead of each agent run step. If you are using the decorator, you can optionally provide a `per_run_step=False` argument to indicate that the toolset only needs to be built once for the entire run.

		When using [Temporal durable execution](./durable_execution/temporal.md), the decorator also accepts an `id` parameter to uniquely identify the toolset. If not provided, the function name is used as the ID.

		# --- DynamicToolset / @agent.toolset tests ---


		def get_dynamic_weather(location: str) -> str:



		@dataclass
		class ToolsetRunStep(TypedDict, Generic[AgentDepsT]):

Add DynamicToolset support in Temporal #3682

Are you sure you want to change the base?

Add DynamicToolset support in Temporal #3682

Conversation

dsfaccini commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Add `DynamicToolset` support in Temporal #3682

Add `DynamicToolset` support in Temporal #3682

dsfaccini commented Dec 9, 2025 •

edited

Loading