Skip to content

UN-3034 [FIX] Add retry backoff configuration for LLMWhisperer client#1836

Open
gaya3-zipstack wants to merge 34 commits intomainfrom
fix/llmwhisperer-retry
Open

UN-3034 [FIX] Add retry backoff configuration for LLMWhisperer client#1836
gaya3-zipstack wants to merge 34 commits intomainfrom
fix/llmwhisperer-retry

Conversation

@gaya3-zipstack
Copy link
Contributor

@gaya3-zipstack gaya3-zipstack commented Mar 6, 2026

What

  • Add configurable retry backoff parameters for the LLMWhisperer v2 client to handle transient HTTP errors (429, 5xx)
  • Increment LLMWhisperer client version in sdk1

Why

  • LLMWhisperer API calls can fail due to transient HTTP errors (rate limiting 429, server errors 5xx)
  • Without retry logic, these transient failures cause document extraction to fail unnecessarily
  • Configurable backoff allows tuning retry behavior per deployment environment

How

  • Added three new environment variables: ADAPTER_LLMW_MAX_RETRIES, ADAPTER_LLMW_RETRY_MIN_WAIT, ADAPTER_LLMW_RETRY_MAX_WAIT
  • Added corresponding constants in WhispererEnv and WhispererDefaults
  • Pass retry parameters (max_retries, retry_min_wait, retry_max_wait) to LLMWhispererClientV2 constructor
  • Updated sample.env files for both backend and prompt-service
  • Incremented LLMWhisperer client version in sdk1

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

  • No. The retry parameters have sensible defaults (3 retries, 1s min wait, 60s max wait) that match previous behavior. Existing deployments without these env vars will use the defaults.

Database Migrations

  • None

Env Config

  • ADAPTER_LLMW_MAX_RETRIES (default: 3) - Max retry attempts for transient HTTP errors. Set 0 to disable.
  • ADAPTER_LLMW_RETRY_MIN_WAIT (default: 1.0) - Min backoff wait in seconds between retries
  • ADAPTER_LLMW_RETRY_MAX_WAIT (default: 60.0) - Max backoff wait in seconds between retries

Relevant Docs

Related Issues or PRs

  • UN-3034

Dependencies Versions

  • Updated llmwhisperer-client version in sdk1

Notes on Testing

  • Verified retry parameters are correctly passed to the LLMWhisperer client constructor
  • Tested with default env values and custom overrides for extraction

Screenshots

image

Checklist

I have read and understood the Contribution Guidelines.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 6, 2026

Quality Gate Failed Quality Gate failed

Failed conditions
12 Security Hotspots
D Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant