Quantcast
Channel: Teradata Forums - Database
Viewing all articles
Browse latest Browse all 14773

Precision loss during expression evaluation - data type issue? - forum topic by tstrick4

$
0
0

Hi, when I execute the following SQL

 sel doc_id
	, prd_id
	, prd_cmn_name
	, net_quant
	, mvt_gravity
	, cast(net_quant*42*(mvt_gravity*8.338426855)-0.0101578 as DECIMAL(19,13))
from monroe_prd.movements 

I am getting the following error:
SELECT Failed 2614: Precision loss during expression evaluation
Relevant table DDL is below

CREATE SET TABLE PRD.MOVEMENTS
     (
      DOC_ID DECIMAL(12,0) TITLE 'SRA Document ID',
      PRD_ID CHAR(4) CHARACTER SET LATIN NOT CASESPECIFIC TITLE 'Product Code',
      PRD_CMN_NAME VARCHAR(30) CHARACTER SET LATIN NOT CASESPECIFIC TITLE 'Product Name',
      NET_QUANT DECIMAL(9,2) TITLE 'Net Quantity',
      GROSS_QUANT DECIMAL(9,2) TITLE 'Gross Quantity',
      MVT_GRAVITY DECIMAL(7,6) TITLE 'Gravity Spec',
      MVT_API DECIMAL(10,7) TITLE 'Movement API',
      CREATEDATETIME TIMESTAMP(0) TITLE 'CreateDate',
      UPDATEDATETIME TIMESTAMP(0) TITLE 'UpdateDate')
UNIQUE PRIMARY INDEX ( DOC_ID ,DOC_NUM )
INDEX ( PRD_ID ,MVT_DATE );

After reviewing the Messages manual, I have tried casting the result as 'FLOAT' and many other combinations of DECIMAL(m,n) and get the same error.  I did the equation in a graphing calculator for a specific row and came up with a DECIMAL(19,13) answer, which is why i've been trying to cast as that data type.  Ultimately I'd like to round the answer to 4 decimal places, but I have not yet gotten to that point.
Thanks in advance for any ideas that you have

Forums: 

Viewing all articles
Browse latest Browse all 14773

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>