Skip to content

Comments

Added support of ApacheArrow batches for BulkUpsert#593

Merged
alex268 merged 9 commits intoydb-platform:release_v2.4.0from
alex268:release_v2.4.0
Feb 20, 2026
Merged

Added support of ApacheArrow batches for BulkUpsert#593
alex268 merged 9 commits intoydb-platform:release_v2.4.0from
alex268:release_v2.4.0

Conversation

@alex268
Copy link
Member

@alex268 alex268 commented Feb 18, 2026

No description provided.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Apache Arrow batch support for executeBulkUpsert by introducing an extensible BulkUpsertData abstraction and an ApacheArrowWriter to build Arrow-encoded batches, with unit/integration tests to validate schema mapping and end-to-end upserts.

Changes:

  • Introduce BulkUpsertData as an interface with concrete implementations for Proto rows, CSV, and Arrow batches.
  • Add ApacheArrowWriter for building Arrow schema + record batches for bulk upsert.
  • Add/adjust unit + integration tests and Maven dependencies/profiles for Apache Arrow.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
table/src/main/java/tech/ydb/table/query/ApacheArrowWriter.java New Arrow writer for encoding YDB types into Arrow vectors + IPC schema/batch serialization.
table/src/main/java/tech/ydb/table/query/BulkUpsertData.java Refactors bulk upsert payload into an interface with factories.
table/src/main/java/tech/ydb/table/query/BulkUpsertProtoData.java New implementation for protobuf TypedValue rows payload.
table/src/main/java/tech/ydb/table/query/BulkUpsertArrowData.java New implementation for Arrow batch payload (schema + data).
table/src/main/java/tech/ydb/table/query/BulkUpsertCsvData.java New implementation for CSV payload.
table/src/main/java/tech/ydb/table/utils/LittleEndian.java Small Javadoc enhancement for bswap(long).
table/src/main/java/tech/ydb/table/Session.java Updates bulk upsert default method to use new BulkUpsertData factories.
table/src/test/java/tech/ydb/table/query/ApacheArrowWriterTest.java New unit tests for Arrow schema/type mapping and validation behavior.
table/src/test/java/tech/ydb/table/integration/BulkUpsertTest.java New integration tests covering Proto and Arrow bulk upserts against YDB.
table/src/test/java/tech/ydb/table/integration/AllTypesRecord.java New test helper generating random records and writing/asserting rows for multiple types.
table/src/test/java/tech/ydb/table/integration/ReadTableTest.java Adapts bulk upsert test setup to new BulkUpsertData API.
table/src/test/java/tech/ydb/table/integration/ValuesReadTest.java Removes a stray debug println.
table/pom.xml Adds optional Arrow deps and a JDK profile for Arrow-related JVM args.
pom.xml Adds Arrow version/deps in dependencyManagement and tweaks JDK profile properties.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@alex268 alex268 changed the title Added support of ApacheArrrow batches for BulkUpsert Added support of ApacheArrow batches for BulkUpsert Feb 18, 2026
@codecov-commenter
Copy link

codecov-commenter commented Feb 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.14%. Comparing base (8fd69a5) to head (ca0d8cb).

Additional details and impacted files
@@                 Coverage Diff                  @@
##             release_v2.4.0     #593      +/-   ##
====================================================
+ Coverage             66.36%   67.14%   +0.77%     
- Complexity             2865     2907      +42     
====================================================
  Files                   345      348       +3     
  Lines                 14430    14751     +321     
  Branches               1492     1523      +31     
====================================================
+ Hits                   9577     9905     +328     
+ Misses                 4189     4181       -8     
- Partials                664      665       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alex268 alex268 merged commit 100d267 into ydb-platform:release_v2.4.0 Feb 20, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants