From: Dean Rasheed <dean.a.rasheed@gmail.com>
Date: Wed, 6 Oct 2021 12:23:13 +0000 (+0100)
Subject: Fix corner-case loss of precision in numeric_power().
X-Git-Tag: REL_10_19~32
X-Git-Url: http://git.postgresql.org/gitweb/?a=commitdiff_plain;h=4853baaaca4a3469247027f7dc86b516e35f7c0a;p=postgresql.git

Fix corner-case loss of precision in numeric_power().

This fixes a loss of precision that occurs when the first input is
very close to 1, so that its logarithm is very small.

Formerly, during the initial low-precision calculation to estimate the
result weight, the logarithm was computed to a local rscale that was
capped to NUMERIC_MAX_DISPLAY_SCALE (1000). However, the base may be
as close as 1e-16383 to 1, hence its logarithm may be as small as
1e-16383, and so the local rscale needs to be allowed to exceed 16383,
otherwise all precision is lost, leading to a poor choice of rscale
for the full-precision calculation.

Fix this by removing the cap on the local rscale during the initial
low-precision calculation, as we already do in the full-precision
calculation. This doesn't change the fact that the initial calculation
is a low-precision approximation, computing the logarithm to around 8
significant digits, which is very fast, especially when the base is
very close to 1.

Patch by me, reviewed by Alvaro Herrera.

Discussion: https://postgr.es/m/CAEZATCV-Ceu%2BHpRMf416yUe4KKFv%3DtdgXQAe5-7S9tD%3D5E-T1g%40mail.gmail.com
---

diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c
index 82bdc755331..8b97e0c1705 100644
--- a/src/backend/utils/adt/numeric.c
+++ b/src/backend/utils/adt/numeric.c
@@ -8134,9 +8134,13 @@ power_var(NumericVar *base, NumericVar *exp, NumericVar *result)
 	 */
 	ln_dweight = estimate_ln_dweight(base);
 
+	/*
+	 * Set the scale for the low-precision calculation, computing ln(base) to
+	 * around 8 significant digits.  Note that ln_dweight may be as small as
+	 * -SHRT_MAX, so the scale may exceed NUMERIC_MAX_DISPLAY_SCALE here.
+	 */
 	local_rscale = 8 - ln_dweight;
 	local_rscale = Max(local_rscale, NUMERIC_MIN_DISPLAY_SCALE);
-	local_rscale = Min(local_rscale, NUMERIC_MAX_DISPLAY_SCALE);
 
 	ln_var(base, &ln_base, local_rscale);
 
diff --git a/src/test/regress/expected/numeric.out b/src/test/regress/expected/numeric.out
index 32ca24892a3..e2ce164ddb8 100644
--- a/src/test/regress/expected/numeric.out
+++ b/src/test/regress/expected/numeric.out
@@ -1606,6 +1606,12 @@ select coalesce(nullif(0.9999999999 ^ 23300000000000, 0), 0) as rounds_to_zero;
               0
 (1 row)
 
+select round(((1 - 1.500012345678e-1000) ^ 1.45e1003) * 1e1000);
+                          round                           
+----------------------------------------------------------
+ 25218976308958387188077465658068501556514992509509282366
+(1 row)
+
 -- cases that used to error out
 select 0.12 ^ (-25);
                  ?column?                  
diff --git a/src/test/regress/sql/numeric.sql b/src/test/regress/sql/numeric.sql
index a6a82f60900..93ef86640d0 100644
--- a/src/test/regress/sql/numeric.sql
+++ b/src/test/regress/sql/numeric.sql
@@ -899,6 +899,7 @@ select 1.2 ^ 345;
 select 0.12 ^ (-20);
 select 1.000000000123 ^ (-2147483648);
 select coalesce(nullif(0.9999999999 ^ 23300000000000, 0), 0) as rounds_to_zero;
+select round(((1 - 1.500012345678e-1000) ^ 1.45e1003) * 1e1000);
 
 -- cases that used to error out
 select 0.12 ^ (-25);