-
Notifications
You must be signed in to change notification settings - Fork 2.9k
fix: use TIMESTAMP WITH TIME ZONE for PostgreSQL in PreciseTimestamp #4388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
fix: use TIMESTAMP WITH TIME ZONE for PostgreSQL in PreciseTimestamp #4388
Conversation
Summary of ChangesHello @vietnamesekid, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request resolves a critical Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request correctly fixes a crash that occurs when using timezone-aware datetime objects with PostgreSQL. The fix involves specifying TIMESTAMP WITH TIME ZONE for the PreciseTimestamp type decorator when the dialect is PostgreSQL. The change is logical and well-targeted. I have one minor suggestion to improve the code structure for better maintainability. Additionally, it would be beneficial to add integration tests with a PostgreSQL backend to verify this fix and prevent future regressions.
|
Response from ADK Triaging Agent Hello @vietnamesekid, thank you for creating this PR! To help reviewers evaluate your contribution, could you please provide the following information as outlined in our contribution guidelines:
This information will help us review your PR more efficiently. Thanks! |
df0914c to
0c74725
Compare
The `PreciseTimestamp` TypeDecorator uses `DateTime` (without timezone) as its default implementation. When used with PostgreSQL + asyncpg, this causes the SQL query to cast parameters as `TIMESTAMP WITHOUT TIME ZONE`, which conflicts with timezone-aware datetime objects (those with `tzinfo=UTC`) that ADK now creates internally (since v1.24.0). This results in the asyncpg error: "can't subtract offset-naive and offset-aware datetimes" The fix adds a PostgreSQL-specific dialect implementation that uses `postgresql.TIMESTAMP(timezone=True)`, consistent with how MySQL already has its own dialect-specific handling (`mysql.DATETIME(fsp=6)`). Fixes google#1848
Adds a migration script that converts TIMESTAMP WITHOUT TIME ZONE
columns to TIMESTAMP WITH TIME ZONE for existing PostgreSQL databases.
Usage:
python -m google.adk.sessions.migration.migrate_postgresql_timestamptz \
--db_url postgresql+asyncpg://user:pass@host:port/dbname
The script checks each ADK table column, skips non-PostgreSQL databases
and columns already using TIMESTAMPTZ, and migrates only what's needed.
83b973c to
ba39e55
Compare
…ution Verifies that PreciseTimestamp returns the correct SQLAlchemy type for each database dialect: - PostgreSQL: TIMESTAMP(timezone=True) to prevent asyncpg offset-naive vs offset-aware datetime errors - MySQL: DATETIME(fsp=6) for microsecond precision - SQLite: default DateTime fallback Includes regression test proving the fix catches the bug: without the PostgreSQL branch, load_dialect_impl returns plain DateTime (timezone=False), which causes asyncpg DataError.
…on-SQLite dialects Fix timezone-naive datetime creation in database_session_service.py that causes incorrect timestamps on servers not running in UTC. Changes: - append_event: use datetime.fromtimestamp(ts, timezone.utc) for non-SQLite dialects when setting session update_time - get_session: use datetime.fromtimestamp(ts, timezone.utc) for non-SQLite dialects when filtering events by after_timestamp Why only database_session_service.py and not schemas/v0.py or v1.py: The schema-level from_event() methods (StorageEvent.from_event) are dialect-agnostic — they have no access to the database engine to know which dialect is in use. Changing them to produce UTC-aware datetimes breaks SQLite roundtrip: SQLite stores datetimes as naive strings, so an aware datetime gets its tzinfo stripped on write, but .timestamp() on the naive readback interprets it as local time, producing wrong values on non-UTC servers (e.g. UTC+7: epoch 3 → stored as 1970-01-01 00:00:03 → read back as naive → .timestamp() = -28797). The service layer (database_session_service.py) is the correct place for this fix because it has access to self.db_engine.dialect.name and can branch on SQLite vs PostgreSQL/MySQL accordingly.
Thanks for sharing your perspective. I agree that without better migration support (#3343), schema related changes are tough to roll out in production. What i was trying to address here is a correctness issue where timestamps depend on the server’s local timezone and can silently drift on non UTC setups. #4365 avoids the immediate error, but this change is mainly about making that behavior more explicit and predictable, scoped to the service layer where the dialect context is available. Totally open to feedback on whether this is the right trade-off at this stage |
am also running into this issue in production, which is why i put these PRs together. I upgraded to 1.24.0 earlier today, but it didn’t resolve the problem on our end 😂 |
|
+1 here for 1.24 version |
Summary
The
PreciseTimestampTypeDecorator usesDateTime(without timezone) as its default implementation. When used with PostgreSQL + asyncpg, this causes the SQL query to cast parameters asTIMESTAMP WITHOUT TIME ZONE, which conflicts with timezone-aware datetime objects (those withtzinfo=UTC) that ADK now creates internally (since v1.24.0).This results in the asyncpg error:
Associated Issue
Related to #1848
Root Cause
PreciseTimestamp.load_dialect_impl()only has special handling for MySQL (mysql.DATETIME(fsp=6)), falling back to plainDateTimefor all other dialects including PostgreSQL.DateTimemaps toTIMESTAMP WITHOUT TIME ZONEin PostgreSQL.DatabaseSessionServicecreates datetime objects withtzinfo=datetime.timezone.utc(offset-aware).TIMESTAMP WITHOUT TIME ZONEcolumns.Fix
Add PostgreSQL-specific dialect implementation using
postgresql.TIMESTAMP(timezone=True), consistent with how MySQL already has its own dialect-specific handling.Testing Plan
tests/unittests/sessions/) — 109 tests passed, 0 failedPreciseTimestamponly returnsTIMESTAMP(timezone=True)when dialect ispostgresql, SQLite falls back to defaultDateTimepostgresql+asyncpg://connection string — session creation, event appending, and session retrieval all work correctly after the fixVerification
Before fix — session creation crashes:
After fix — SQL now correctly uses
TIMESTAMP WITH TIME ZONE:Unit tests:
$ python3 -m pytest tests/unittests/sessions/ -v
======================== 109 passed, 1 warning in 2.67s ========================
Database Migration Required
Users with existing PostgreSQL databases need to run:
A Python migration script is also included: src/google/adk/sessions/migration/migrate_postgresql_timestamptz.py