Skip to content

[ES|QL] Substitue date_trunc with round_to when the pre-calculated rounding points are available #128639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

fang-xing-esql
Copy link
Member

@fang-xing-esql fang-xing-esql commented May 29, 2025

This is a subtask of the filter-by-filter aggregation in ES|QL. And the following changes are included in this PR.

  • Consolidated min/max per shard into SearchStats, according to the statistics retrieved from Lucent for DateFieldType only, added SearchContextStatsTests, and tested min/max.
  • At date nodes, if min/max are available in SearchStats, provide them to DateTrunc.createRounding, call prepare(min, max), instead of prepareForUnknown().
  • At data node, pre-calculated rounding points are returned by fixedRoundingPoints(), substitute date_trunc and bucket with round_to function. A new rule SubstituteSurrogateExpressionsWithSearchStats is added in the local rewrite batch in LocalLogicalPlanOptimizer to do the substitution.
  • The rewrite from date_trunc/bucket to round_to does not apply to date_nanos yet, as the underlying APIs support millis only, and nanos has not been tested yet.
  • PreparedRounding.maybeUseArray may hit an assert if the calendar intervals are multiple quantities, like 10 month or 3 quarter, date histogram aggregation has this limitation documented here,this is likely why the assert hasn't been hit before, and the same limitation will apply to this rewrite.

Next step: performance validations.

@elasticsearchmachine
Copy link
Collaborator

Hi @fang-xing-esql, I've created a changelog YAML for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants