[Feature](scan) add variable adjust_tablet_reader_batch_size_by_limit#61003
[Feature](scan) add variable adjust_tablet_reader_batch_size_by_limit#61003BiteTheDDDDt wants to merge 1 commit intoapache:masterfrom
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
There was a problem hiding this comment.
Pull request overview
This PR introduces a new session/query option to optionally size the OLAP tablet reader’s batch size based on the query LIMIT, aiming to reduce scan work for queries with small limits (especially when rows are wide).
Changes:
- Added
adjust_tablet_reader_batch_size_by_limittoTQueryOptions(Thrift) and to FE session variables (forwarded to BE). - Added a
RuntimeStateaccessor for the new option in BE. - Updated
OlapScanner::prepare()to optionally set tablet reader batch size tomin(batch_size, limit)when enabled.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| gensrc/thrift/PaloInternalService.thrift | Adds a new optional TQueryOptions boolean field to carry the flag from FE to BE. |
| fe/fe-core/src/main/java/org/apache/doris/qe/SessionVariable.java | Defines the session variable and forwards it into TQueryOptions via toThrift(). |
| be/src/runtime/runtime_state.h | Adds a helper to read the new option from TQueryOptions safely. |
| be/src/vec/exec/scan/olap_scanner.cpp | Applies the option by adjusting BlockReader batch size based on scanner limit. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
FE Regression Coverage ReportIncrement line coverage |
What problem does this PR solve?
after #22240, we do not push limit to tablet reader.
but sometime the row data is big, so we need use limit to scan less data.
adjust_tablet_reader_batch_size_by_limit=false;
adjust_tablet_reader_batch_size_by_limit=true;
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)