Skip to content

fix: add wall-clock certificate expiry check to webhook TLS rotation#1175

Merged
razvan merged 5 commits intomainfrom
fix/webhook-tls-wall-clock-rotation
Mar 17, 2026
Merged

fix: add wall-clock certificate expiry check to webhook TLS rotation#1175
razvan merged 5 commits intomainfrom
fix/webhook-tls-wall-clock-rotation

Conversation

@lfrancke
Copy link
Member

Description

The rotation interval uses tokio's monotonic clock, but certificate validity uses wall-clock time. When these diverge (hibernation, VM migration, cgroup freezing), the certificate can expire before rotation.

Add a periodic wall-clock check (every 5 minutes) that compares SystemTime::now() against the certificate's not_after field and triggers early rotation if the cert is within 4 hours of expiry.

Fixes: #1174

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Reviewer

  • Code contains useful comments
  • Code contains useful logging statements
  • (Integration-)Test cases added
  • Documentation added or updated. Follows the style guide.
  • Changelog updated
  • Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

  • Proper release label has been added

@lfrancke lfrancke force-pushed the fix/webhook-tls-wall-clock-rotation branch 6 times, most recently from 9e714ab to a93a7f6 Compare March 12, 2026 10:11
The rotation interval uses tokio's monotonic clock, but certificate
validity uses wall-clock time. When these diverge (hibernation, VM
migration, cgroup freezing), the certificate can expire before rotation.

Add a periodic wall-clock check (every 5 minutes) that compares
SystemTime::now() against the certificate's not_after field and triggers
early rotation if the cert is within 4 hours of expiry.

Fixes: #1174
Remove the monotonic 20h rotation interval and the supplementary
wall-clock check. Instead, use a single periodic check (every 5 min)
that compares wall-clock time against the certificate's not_after.

Also derive the expiry buffer from the certificate lifetime (1/6) so
it scales if the lifetime ever changes, and add comments documenting
the relationship between lifetime and check interval.
@lfrancke lfrancke force-pushed the fix/webhook-tls-wall-clock-rotation branch from a93a7f6 to 003a628 Compare March 12, 2026 10:15
@razvan
Copy link
Member

razvan commented Mar 16, 2026

Tested the secret operator against this branch and it looks good (it doesn't introduce additional breakage):

--- PASS: kuttl (167.86s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/tls_openshift-false_rsa-key-length-2048_custom-secret-names-False (11.65s)
        --- PASS: kuttl/harness/listener_openshift-false (27.88s)
        --- PASS: kuttl/harness/kerberos_krb5-1.21.1_openshift-false (67.90s)
        --- PASS: kuttl/harness/tls-truststore_openshift-false_truststore-target-kind-Secret (7.65s)
        --- PASS: kuttl/harness/tls-truststore_openshift-false_truststore-target-kind-ConfigMap (8.19s)
        --- PASS: kuttl/harness/tls_openshift-false_rsa-key-length-3072_custom-secret-names-True (15.86s)
        --- PASS: kuttl/harness/cert-manager-tls_openshift-false (64.00s)
        --- PASS: kuttl/harness/tls_openshift-false_rsa-key-length-2048_custom-secret-names-True (20.59s)
        --- PASS: kuttl/harness/tls_openshift-false_rsa-key-length-3072_custom-secret-names-False (26.58s)
        --- PASS: kuttl/harness/non-sensitive-data (43.71s)
PASS

@razvan razvan self-requested a review March 16, 2026 16:50
Copy link
Member

@razvan razvan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A changelog entry is missing and I made a small suggestion but otherwise very nice.

@Techassi Techassi self-requested a review March 16, 2026 17:08
@razvan razvan marked this pull request as ready for review March 17, 2026 10:10
@razvan razvan added this pull request to the merge queue Mar 17, 2026
Merged via the queue into main with commit 7e713a4 Mar 17, 2026
5 checks passed
@razvan razvan deleted the fix/webhook-tls-wall-clock-rotation branch March 17, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Webhook TLS certificate rotation uses monotonic time, but cert validity uses wall-clock time

3 participants