Conversation
|
|
||
| ## Metrics | ||
|
|
||
| Use the Tally library (`go.temporal.io/sdk/contrib/tally`) with Prometheus: |
There was a problem hiding this comment.
Why recommend tally here over otel?
There was a problem hiding this comment.
otel was cut from scope of initial skill launch (#23), since it was still in flux.
Tally & Prometheus is simple enough to include since its more narrowly scoped to metrics only. All other SDK languages in the skill have included metrics, but (for now) excluded tracing.
references/go/data-handling.md
Outdated
|
|
||
| ## Custom Data Converter | ||
|
|
||
| Implement the `converter.DataConverter` interface (`ToPayload`, `FromPayload`, `ToPayloads`, `FromPayloads`, `ToString`, `ToStrings`) or compose a new converter from existing ones. |
There was a problem hiding this comment.
More of a meta question to skills in general (as someone not super up-to-speed), this line reminds me that some of these docs are basically the same as the code itself, and now introduce a new place we have to keep in sync with the code itself, like if we happen to add a new method to this interface. I think someone could argue it's equally as fast to look at the code itself to see what methods the interface has.
Is this skill basically organizing the codebase in a AI digestible way so it doesn't need to dig through the code itself? Is adding information like this proven to be beneficial to the skill? Would the gain from this be comparable to if we ensured our docs were thorough and up to date?
There was a problem hiding this comment.
Lots of questions here, I'll do what i can to break it down :).
Q1: What value does a skill provide vs. docs vs. the SDK code itself?
For reference, you can currently use these 2 MCP servers:
- Kapa: MCP server for docs access.
- context7: MCP server for SDK API search questions
So the question is, what value does a skill provide compared to those two MCP servers?
Primarily, its about providing a more tightly curated "garden path" to show to the agent, compared to the much broader in scope docs and SDK code. The aim is to make it both more digestable (= faster for the agent to index into the content that matters), and be able to force items into its context.
As an example, consider the data converters. Just by loading the top level SKILL.md, the agent will (no choice) get links to references/go/data-handling.md in its context. If its doing some task where data serialization seems relevant, it would be highly likely to then load references/go/data-handling.md, since it knows it exists. If you are just using Kapa or context7, there is nothing in the agent's context that alerts the agent that Temporal has any concept of data handling / serialization / etc., so it might not decide to do an MCP call to lookup in the docs or SDK code about data conversion.
Q2: How do we know this is beneficial to the agent?
The only way we will know for sure is by doing extensive evaluations. I'm currently working on this, but if you want to help, the most helpful thing you can do is note down Temporal coding tasks that a coding agent performed poorly on. Given difficult tasks, we can then observe that the agent completes them successfully given the skill. Let me know if you know of any tricky tasks for agents!
Q3: How do we keep skill in sync?
This is discussed a bit in the PRD, but essentially I will track what SDK versions the skill was currently written for, and then in the future can diff changelogs, and identify and fix stale areas of the skill. I'd also like to investigate snipsync sorts of stuff for managing code snippets better.
There was a problem hiding this comment.
That being said, I don't like this Custom Data Converter content thats here. I don't think its descriptive enough to really be helpful to the agent. Will fix.
There was a problem hiding this comment.
Has been changed to a code sample.
There was a problem hiding this comment.
Thanks for the answers! I'll take a closer look at the PRD as well
if you want to help, the most helpful thing you can do is note down Temporal coding tasks that a coding agent performed poorly on.
sounds good, will keep an eye out and remember to give you feedback when I come across this 😃
What was changed
Adds support for Go.
This PR will be merged instead of #6, which is the much older and way less complete version of Go. Once approved I will force-push these commits to the branch of #6, in case anyone was actually using #6 for testing.