feat(occurrences on eap): Implement EAP read for error counts in suspect tag calculation

[shashjar]

Using an EndpointTraceItemTable RPC query to fetch error counts from EAP, used in the calculation of suspect tags.

Tested locally to ensure that the count aggregation is coming back correctly from EAP:

sentry.issues.suspect_tags: >>>>> query_error_counts called (organization_id=1 project_id=2 start='2025-12-13 00:00:00+00:00' end='2025-12-13 01:00:00+00:00' environments=[])

sentry.utils.snuba_rpc: Running a EndpointTraceItemTable RPC query (rpc_query={'meta': {'organizationId': '1', 'cogsCategory': 'issues', 'referrer': 'issues.suspect_tags.query_error_counts', 'projectIds': ['2'], 'startTimestamp': '2025-12-13T00:00:00Z', 'endTimestamp': '2025-12-13T01:00:00Z', 'traceItemType': 'TRACE_ITEM_TYPE_OCCURRENCE'}, 'columns': [{'aggregation': {'aggregate': 'FUNCTION_COUNT', 'key': {'type': 'TYPE_INT', 'name': 'group_id'}}, 'label': 'count'}], 'limit': 1} referrer='issues.suspect_tags.query_error_counts' organization_id=1 trace_item_type=7)

sentry.issues.suspect_tags: >>>>> EAP response received (response_count=1 has_column_values=True raw_response='column_values {\n  attribute_name: "count"\n  results {\n    val_double: 6\n  }\n}\npage_token {\n  offset: 1\n}\nmeta {\n  request_id: "aa3ed9bf554a40099ab2a70cc9b097a9"\n  query_info {\n    stats {\n      progress_bytes: 486\n    }\n    metadata {\n    }\n  }\n  downsampled_storage_meta {\n  }\n}\n')

sentry.issues.suspect_tags: >>>>> Snuba vs EAP comparison (snuba_count=6 eap_count=6 match=True)

To roll this out, we can:

1. Merge in this PR (no change in behavior in any region)
2. Flip the eap.occurrences.should_double_read option on for the S4S region only
3. At this point, we'll be calculating both values in S4S but sticking with the Snuba value as the source of truth
4. We should see the eap.occurrences.validate_reads metric start getting recorded and then converge to 0 mismatches over time (as EAP gets up-to-speed on all the groups within retention)
5. At that point (01/21/2026, 90 days after 10/23/2025), we should be able to remove the legacy read path and use EAP only

[codecov]

Codecov Report

❌ Patch coverage is 91.30435% with 2 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/sentry/issues/suspect_tags.py	91.30%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #104916      +/-   ##
===========================================
- Coverage   80.57%    80.49%   -0.08%     
===========================================
  Files        9431      9416      -15     
  Lines      404261    403719     -542     
  Branches    25655     25756     +101     
===========================================
- Hits       325718    324990     -728     
- Misses      78075     78295     +220     
+ Partials      468       434      -34     

[thetruecpaul]

LGTM!