Support decimal literal with Calcite #3673

LantaoJin · 2025-05-26T07:35:07Z

Description

Support decimal literal with Calcite

Behavior in v2 is no changing.
The decimal number, e.g.. 3.0, 2.5 in PPL statement will be treated as a decimal literal unless adding a d(double), f(float) suffix
This decimal literal will keep Decimal type in Calcite plan.
The decimal literal is present as double in runtime (ExprType)

Related Issues

Resolves #3614

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Lantao Jin <[email protected]>

LantaoJin · 2025-05-26T07:38:25Z

core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java

@@ -166,7 +166,7 @@ void populate() {
      registerOperator(ADD, SqlStdOperatorTable.PLUS);
      registerOperator(SUBTRACT, SqlStdOperatorTable.MINUS);
      registerOperator(MULTIPLY, SqlStdOperatorTable.MULTIPLY);
-      // registerOperator(DIVIDE, SqlStdOperatorTable.DIVIDE);
+      registerOperator(TRUNCATE, SqlStdOperatorTable.TRUNCATE);


This is a missing function which is not related to this PR. But we need it in CalciteMathematicalFunctionIT.java

LantaoJin · 2025-05-26T07:41:10Z

...st/java/org/opensearch/sql/calcite/standalone/CalcitePPLCryptographicFunctionPushdownIT.java

+
+import org.opensearch.sql.common.setting.Settings;
+
+public class CalcitePPLCryptographicFunctionPushdownIT extends CalcitePPLCryptographicFunctionIT {


missing IT which is not related to this issue.

LantaoJin · 2025-05-26T07:42:25Z

.../java/org/opensearch/sql/calcite/standalone/CalcitePPLDateTimeBuiltinFunctionPushdownIT.java

+public class CalcitePPLDateTimeBuiltinFunctionPushdownIT
+    extends CalcitePPLDateTimeBuiltinFunctionIT {


incorrect inheritance which is not related to this issue.

LantaoJin · 2025-05-26T07:42:39Z

...g-test/src/test/java/org/opensearch/sql/calcite/standalone/CalcitePPLFillnullPushdownIT.java

+
+import org.opensearch.sql.common.setting.Settings;
+
+public class CalcitePPLFillnullPushdownIT extends CalcitePPLFillnullIT {


missing IT which is not related to this issue.

LantaoJin · 2025-05-26T07:42:48Z

...test/src/test/java/org/opensearch/sql/calcite/standalone/CalcitePPLIPFunctionPushdownIT.java

+
+import org.opensearch.sql.common.setting.Settings;
+
+public class CalcitePPLIPFunctionPushdownIT extends CalcitePPLIPFunctionIT {


LantaoJin · 2025-05-26T07:44:35Z

integ-test/src/test/java/org/opensearch/sql/ppl/MathematicalFunctionIT.java

+    if (isCalciteEnabled()) {
+      verifySchema(result, schema("f", null, "int"));
+    } else {
+      verifySchema(result, schema("f", null, "bigint"));
+    }


return types between v2 and v3 are different, not related to this issue.

Why dose this IT could pass before?

Why dose this IT could pass before?

CalciteMathematicalFunctionIT was missing before this PR.

Signed-off-by: Lantao Jin <[email protected]>

qianheng-aws · 2025-05-30T08:03:18Z

core/src/main/java/org/opensearch/sql/ast/expression/Literal.java

 public class Literal extends UnresolvedExpression {

  private final Object value;
  private final DataType type;

+  public Literal(Object value, DataType dataType) {
+    if (dataType == DataType.DECIMAL && value instanceof Double) {


When will we create a decimal literal with double value?

This is to be compatible with calls like new Literal(2.0, Decimal) , otherwise we can only use new Literal(new BigDecimal("2.0"), Decimal)

penghuo

Thanks for the change! One minor suggestion would be to add a small note in the documentation about this change to help users understand the precision behavior, but that could also be addressed separately.

LantaoJin · 2025-06-04T01:39:39Z

Thanks for the change! One minor suggestion would be to add a small note in the documentation about this change to help users understand the precision behavior, but that could also be addressed separately.

Sure, I will summary the break changes for 3.1.0 release in a new PR.

dai-chen

Thanks for the changes! Should we also update user manualhttps://github.com/opensearch-project/sql/blob/main/docs/user/ppl/general/datatypes.rst?

LantaoJin · 2025-06-04T23:15:17Z

Thanks for the changes! Should we also update user manualhttps://github.com/opensearch-project/sql/blob/main/docs/user/ppl/general/datatypes.rst?

Decimal is not a new supported datatype, we'd better update that doc after #3619 addressed. I will highlight this breaking changes in release notes.

Support decimal literal with Calcite

91309af

Signed-off-by: Lantao Jin <[email protected]>

LantaoJin commented May 26, 2025

View reviewed changes

LantaoJin added 4 commits May 27, 2025 11:55

fix IT

2879bc5

Signed-off-by: Lantao Jin <[email protected]>

fix IT

2740f75

Signed-off-by: Lantao Jin <[email protected]>

add long, short literal

a01dfb3

Signed-off-by: Lantao Jin <[email protected]>

fix UT

f25b88d

Signed-off-by: Lantao Jin <[email protected]>

LantaoJin added breaking calcite calcite migration releated labels May 27, 2025

fix IT

e588158

Signed-off-by: Lantao Jin <[email protected]>

LantaoJin marked this pull request as ready for review May 27, 2025 09:10

LantaoJin requested review from ps48, kavithacm, derek-ho, joshuali925, dai-chen, YANG-DB, mengweieric, Swiddis, penghuo, seankao-az, MaxKsyunz, Yury-Fridlyand, anirudha, forestmvey, acarbonetto, GumpacG, ykmr1224, noCharger and qianheng-aws as code owners May 27, 2025 09:10

Merge remote-tracking branch 'upstream/main' into issues/3614

9cdcafe

qianheng-aws reviewed May 30, 2025

View reviewed changes

This comment was marked as off-topic.

Sign in to view

penghuo approved these changes Jun 1, 2025

View reviewed changes

dai-chen approved these changes Jun 4, 2025

View reviewed changes

LantaoJin merged commit f1e3b5b into opensearch-project:main Jun 4, 2025
22 checks passed

LantaoJin mentioned this pull request Jun 5, 2025

Support ResourceMonitor with Calcite #3738

Merged

7 tasks


		import org.opensearch.sql.common.setting.Settings;

		public class CalcitePPLCryptographicFunctionPushdownIT extends CalcitePPLCryptographicFunctionIT {

		public class CalcitePPLDateTimeBuiltinFunctionPushdownIT
		extends CalcitePPLDateTimeBuiltinFunctionIT {


		import org.opensearch.sql.common.setting.Settings;

		public class CalcitePPLFillnullPushdownIT extends CalcitePPLFillnullIT {


		import org.opensearch.sql.common.setting.Settings;

		public class CalcitePPLIPFunctionPushdownIT extends CalcitePPLIPFunctionIT {

Support decimal literal with Calcite #3673

Support decimal literal with Calcite #3673

Uh oh!

Conversation

LantaoJin commented May 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

penghuo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LantaoJin commented Jun 4, 2025

Uh oh!

dai-chen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

LantaoJin commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

LantaoJin commented May 26, 2025 •

edited

Loading

penghuo left a comment •

edited

Loading

LantaoJin commented Jun 4, 2025 •

edited

Loading