Quantcast
Channel: Teradata Forums - Database
Viewing all articles
Browse latest Browse all 14773

Precision loss during expression evaluation - data type issue? - response (1) by VandeBergB

$
0
0

The 2614 error says that the result is too small.  I plugged some literals your  cast stmt, and got the same errors.  When i force the order of operations and limit the values to four decimal places (you said you wanted to round to four places) it works.
Not sure if you want this evaluated left to right, but ...

SELECT 123456.33*42*(1.888333*8.338426855)-0.0101578

generates a 2614 error, casting that as float generates the error as well.

SELECT CAST(123456.3333*((42.0000*(1.8883*8.998))- 0.01015) AS FLOAT)

extending the integer of 42 to 42.0000 and limiting the other values to four decimal places returns a result....

SELECT CAST(123456.3333*((42.0000*(1.8883*8.338))- 0.0101578) AS FLOAT)

interesting conundrum...

SELECT CAST((123456.3333*42*1.8883*8.3384269) - .0101578 AS FLOAT)

the last row works as well.
What does your data profile look like, for the columns involved in the calculation?


Viewing all articles
Browse latest Browse all 14773

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>