MATLAB: Weird rounding error

errormathematicsrounding

Hello. I'm getting the following odd "error" which I hope is just a display thing but really doesn't seem to be. I'm using R2016a on redhat linux.

>> format long

>> 177.5 – 170.6

ans = 6.900000000000006

>> 177.5 – 170.6 -6.9

ans = 5.329070518200751e-15

>> 1.5 – 0.6

ans = 0.900000000000000

>> 10.5 – 9.6

ans = 0.900000000000000

>> 100.5 – 99.6

ans = 0.900000000000006

>> (100.5 – 99.6) > 0.9

ans = 1

Best Answer

Hi Jason, this is just Computer Science 101. Since computers have only so many bits to describe floating point numbers, there are going to be differences like this (not errors), down at the last-bit level. Good that you had enough interest in the details to recognize this effect. See 'help eps'.

In IEEE 754 standard which Matlab uses, 64 bit floating point numbers have 53 bits allotted for the significant figure part. It's worthwhile to go to Wikipedia for a summary of IEEE 754.

Related Solutions

MATLAB: Improper negative number representation

IEEE double precision floating point cannot represent the number -6e-9 exactly. What you are seeing is the decimal representation of the closest binary floating point bit pattern that IEEE double can represent. This is normal behavior for binary floating point systems. See this link for more discussion:

https://www.mathworks.com/matlabcentral/answers/57444-faq-why-is-0-3-0-2-0-1-not-equal-to-zero?s_tid=ta_ans_results

The "exact" decimal conversion of the binary floating point numbers stored for the numbers in your examples:

>> num2strexact(-1*1e-9)
ans =
    '-1.0000000000000000622815914577798564188970686927859787829220294952392578125e-9'
>> num2strexact(-2*1e-9)
ans =
    '-2.000000000000000124563182915559712837794137385571957565844058990478515625e-9'
>> num2strexact(-3*1e-9)
ans =
    '-3.0000000000000003936399275115964879745433790958486497402191162109375e-9'
>> num2strexact(-4*1e-9)
ans =
    '-4.00000000000000024912636583111942567558827477114391513168811798095703125e-9'
>> num2strexact(-5*1e-9)
ans =
    '-5.0000000000000001046128041506423633766331704464391805231571197509765625e-9'
>> num2strexact(-6*1e-9)
ans =
    '-6.000000000000000787279855023192975949086758191697299480438232421875e-9'
>> num2strexact(-7*1e-9)
ans =
    '-7.00000000000000064276629334271591365013165386699256487190723419189453125e-9'
>> num2strexact(-8*1e-9)
ans =
    '-8.0000000000000004982527316622388513511765495422878302633762359619140625e-9'
>> num2strexact(-9*1e-9)
ans =
    '-9.0000000000000011809197825347894639236301372875459492206573486328125e-9'

MATLAB: 6*0.003125 – 0.018750 = 0, right? How to get the right answer

Matlab is giving you the right answer. Matlab uses IEEE 754 binary floating point double precision. Floating point operations are not algebraic and not generally commutative.

In binary floating point there is no exact representation of 0.1, just the same way that in base 10 there is no way to exactly represent 1/7 in any finite precision decimal number.

Best Answer

Related Solutions

MATLAB: Improper negative number representation

MATLAB: 6*0.003125 – 0.018750 = 0, right? How to get the right answer

Related Question