Implement execution_timeout semantics for AirbyteTriggerSyncOperator in deferrable mode#64051
Draft
SameerMesiah97 wants to merge 1 commit intoapache:mainfrom
Draft
Conversation
c255d11 to
5951be9
Compare
Restore execution_timeout semantics in deferrable mode by propagating timeouts through the trigger and explicitly cancelling Airbyte jobs when the task exceeds its execution deadline. This preserves behavior parity with non-deferrable execution and avoids leaking Airbyte jobs. Add tests covering execution timeout handling in both the operator and trigger, including successful cancellation and best-effort behavior when job cancellation fails.
5951be9 to
52ddbc1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This change enforces
execution_timeoutforAirbyteTriggerSyncOperatorin deferrable mode.Previously, when the operator deferred,
execution_timeoutwas not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out.The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received.
Rationale
In non-deferrable mode,
execution_timeoutis enforced by the scheduler, which terminates the task process and invokeson_kill()to cancel the external Airbyte job.In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective,
execution_timeoutis a hard task-level limit and should behave consistently regardless of execution mode.Without explicit handling in the trigger/operator interaction,
execution_timeoutbecomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior.This is the same class of issue addressed in PR #61472 for
DbtCloudRunJobOperator.Notes
timeoutparameter only controls how long the operator waits for job completion and does not imply cancellation.execution_timeoutandtimeoutare set, the earlier deadline takes precedence.execute_complete, genericAirflowExceptionraises have been replaced with more appropriate, specific exception types.Tests
execution_deadlinefield is always serialized.Documentation
execution_deadlineparameter.timeoutvsexecution_timeout; added an entry forexecution_timeout.Backwards Compatibility
This change does not modify public APIs or method signatures.
Behavior is changed such that Airbyte jobs are now cancelled when
execution_timeoutis reached in deferrable mode. Previously, jobs could continue running after the task timed out.Closes: #64048