------------------------------------------------------------------------------------------------
[BUG/PRB.] ADJUST THE PRECISION OF THE FLOATING-POINT NUMBERS FOR THE ROUND() FUNCTION
January 2024
------------------------------------------------------------------------------------------------
CCB
1. BUG:
There is a test program from Mr. Christof Wollenhaupt:
*PROC testroundfunction
?ROUND(512.9250000000,2) && Displays 512.92
?ROUND(512.925000000,2) && Displays 512.93
WAIT
RETURN
* END OF PROC TESTROUNDFUNCTION.
We think they will display 512.93, but:
?ROUND(512.9250000000,2) && Displays 512.92
2. CAUSE:
Please refer to the head of the testroundfunction.fxp:
00000000: FE F2 FF 22-02 01 00 00-00 D8 00 00-00 A1 00 00
00000010: 00 37 00 00-00 00 00 00-00 00 00 00-00 00 00 00
00000020: 00 00 00 00-00 00 00 3F-27 00 00 00-00 25 00 00
00000030: 00 00 00 00-00 00 00 00-00 68 00 00-00 03 00 00
00000040: 00 62 00 00-00 8F 56 82-47 1C 00 00-00 85 00 00
00000050: 35 00 00 00-19 00 02 F8-03 01 FC 43-FA 0E 0A 66
00000060: 66 66 66 66-07 80 40 F8-01 02 54 FD-FE 19 00 02
00000070: F8 03 01 FC-43 FA 0D 09-66 66 66 66-66 07 80 40
00000080: F8 01 02 54-FD FE 03 00-55 00 00 91-01 91 01 31
00000090: 00 00 00 00-00 00 00 00-00 00 00 00-00 00 00 00
the compiled p-code for the statement "?ROUND(512.9250000000,2)" from 0x00000054:
19 00 02 F8 03 01 FC 43 FA 0E 0A 66 66 66 66 66 07 80 40 F8 01 02 54 FD FE
--------------------------------------------------------------------------
The width of the floating-point number is 0x0E (dec: 14).
The number of decimal places is 0x0A (dec: 10).
The 64-bit floating-point number is 0x4080076666666666.
the compiled p-code for the statement "?ROUND(512.925000000,2)" from 0x0000006D:
19 00 02 F8 03 01 FC 43 FA 0D 09 66 66 66 66 66 07 80 40 F8 01 02 54 FD FE
--------------------------------------------------------------------------
The width of the floating-point number is 0x0D (dec: 13).
The number of decimal places is 0x09 (dec: 9).
The 64-bit floating-point number is 0x4080076666666666.
They are the same 64-bit floating-point numbers:
512.9250000000 = 0x4080076666666666
512.925000000 = 0x4080076666666666
the difference is the width of the floating-point number and the number of decimal places.
For the details when VFP calculates ROUND(512.9250000000,2):
512. 9249 9999 9999 9546 0
+ 0. 0000 0000 0000 0500 0 (n=10, eps = 0.5*10^(-(n+3)) = 5.0000000000000001520E-14)
----------------------------
512. 9249 9999 9999 9546 0
512. 9249 9999 9999 9546 0
* 100. 0000 0000 0000 0000 0
----------------------------
51292. 4999 9999 9992 720
+ 0. 5000 0000 0000 0000 0
----------------------------
51292. 9999 9999 9992 720
floor(51292. 9999 9999 9992 720) = 51292.000000000000000
51292. 0000 0000 0000 0000 0
/ 100. 0000 0000 0000 0000 0
----------------------------
512. 9199 9999 9999 9590 0
+ 0. 0000 0500 0000 0000 0 (n=2, eps = 0.5*10^(-(n+3)) = 5.0000000000000004090E-06)
----------------------------
512. 9200 0499 9999 9464 0
So ROUND(512.9250000000,2) = 512.92.
For the details when VFP calculates ROUND(512.925000000,2):
512. 9249 9999 9999 9546 0
+ 0. 0000 0000 0000 4999 9 (n=9, eps = 0.5*10^(-(n+3)) = 4.9999999999999999000E-13)
----------------------------
512. 9250 0000 0000 4092 0
512. 9250 0000 0000 4092 0
* 100. 0000 0000 0000 0000 0
----------------------------
51292. 5000 0000 0043 660
+ 0. 5000 0000 0000 0000 0
----------------------------
51293. 0000 0000 0043 660
floor(51293. 0000 0000 0043 660) = 51293.000000000000000
51293. 0000 0000 0000 0000 0
/ 100. 0000 0000 0000 0000 0
----------------------------
512. 9299 9999 9999 9500 0
+ 0. 0000 0500 0000 0000 0 (n=2, eps = 0.5*10^(-(n+3)) = 5.0000000000000004090E-06)
----------------------------
512. 9300 0499 9999 9374 0
So ROUND(512.925000000,2) = 512.93.
If we enable adjusting the precision of the floating-point numbers in Visual FoxPro Advanced,
Visual FoxPro Advanced will set 15 valid digits for the following functions:
ROUND() Function.
Visual FoxPro Advanced will adjust the floating-point numbers before call the floor() function:
51292. 9999 9999 9992 720
+00000. 0000 0000 005
--------------------------
51293. 0000 0000 0042 720
floor(51292. 9999 9999 9992 720 +00000. 0000 0000 005) = 51293.000000000000000
So ROUND(512.9250000000,2) = 512.93.
51293. 0000 0000 0043 660
+00000. 0000 0000 005
--------------------------
51293. 0000 0000 0093 660
floor(51293. 0000 0000 0043 660 +00000. 0000 0000 005) = 51293.000000000000000
So ROUND(512.925000000,2) = 512.93.
Visual FoxPro Advanced will also set 15 valid digits and round the value to integer when subtract two datetimes
to get the difference between them in seconds.
For the test program from Mr. Sergey Karimov:
lnsec= {^2000/01/01 01:01:00 AM}-{^2000/01/01 00:00:00 AM}
a1= lnsec%3600
a2= a1/60
?lnsec, a1, a2, FLOOR(a1)/60, FLOOR(a1/60)
Now it can run fine if we enable adjusting the precision of the floating-point numbers in Visual FoxPro Advanced.
3. RESOLUTION:
We can write some code to fix the BUG.
Label4ae942 ::
push ebp ;0x004ae942 : 55
mov ebp , esp ;0x004ae943 : 8bec
and esp , 0FFFFFFF8h ;0x004ae945 : 83e4f8
sub esp , 018h ;0x004ae948 : 83ec18
push ebx ;0x004ae94b : 53
push ebp ;0x004ae94c : 55
push esi ;0x004ae94d : 56
push edi ;0x004ae94e : 57
mov dword ptr [ esp + 20 ] , 00h ;0x004ae94f : c744241400000000
call Fun420d8c ;0x004ae957 : e83024f7ff
mov esi , eax ;0x004ae95c : 8bf0
lea ebx , dword ptr [ esi + 44 ] ;0x004ae95e : 8d5e2c
push ebx ;0x004ae961 : 53
lea edi , dword ptr [ esp + 32 ] ;0x004ae962 : 8d7c2420
call Fun420dd3 ;0x004ae966 : e86824f7ff
test eax , eax ;0x004ae96b : 85c0
jne Label4aeaaf ;0x004ae96d : 0f853c010000
mov edi , ebx ;0x004ae973 : 8bfb
call Fun529a6e ;0x004ae975 : e8f4b00700
cmp byte ptr [ebx] , 049h ;0x004ae97a : 803b49
mov ebp , dword ptr [ ebx + 12 ] ;0x004ae97d : 8b6b0c
mov edi , 0385h ;0x004ae980 : bf85030000
jne Label591cc0 ;0x004ae985 : 0f8535330e00
Label4ae98b ::
mov al , byte ptr [esi] ;0x004ae98b : 8a06
xor ecx , ecx ;0x004ae98d : 33c9
cmp al , 059h ;0x004ae98f : 3c59
sete cl ;0x004ae991 : 0f94c1
test ecx , ecx ;0x004ae994 : 85c9
mov dword ptr [ esp + 24 ] , ecx ;0x004ae996 : 894c2418
jne Label591ce8 ;0x004ae99a : 0f8548330e00
Label4ae9a0 ::
cmp al , 049h ;0x004ae9a0 : 3c49
je Label591cf6 ;0x004ae9a2 : 0f844e330e00
cmp al , 059h ;0x004ae9a8 : 3c59
je Label591d0b ;0x004ae9aa : 0f845b330e00
cmp al , 04Eh ;0x004ae9b0 : 3c4e
jne Label591d26 ;0x004ae9b2 : 0f856e330e00
Label4ae9b8 ::
test ebp , ebp ;0x004ae9b8 : 85ed
jl Label591d4e ;0x004ae9ba : 0f8c8e330e00
mov eax , dword ptr [ esi + 8 ] ;0x004ae9c0 : 8b4608
cmp eax , ebp ;0x004ae9c3 : 3bc5
mov ecx , eax ;0x004ae9c5 : 8bc8
jle Label591d60 ;0x004ae9c7 : 0f8e93330e00
Label4ae9cd ::
cmp ecx , 012h ;0x004ae9cd : 83f912
jnl Label591d6e ;0x004ae9d0 : 0f8d98330e00
cmp eax , ebp ;0x004ae9d6 : 3bc5
jle Label591d67 ;0x004ae9d8 : 0f8e89330e00
mov edx , eax ;0x004ae9de : 8bd0
Label4ae9e0 ::
cmp ebp , 012h ;0x004ae9e0 : 83fd12
jg Label591d78 ;0x004ae9e3 : 0f8f8f330e00
Label4ae9e9 ::
mov ax , word ptr [ esi + 22 ] ;0x004ae9e9 : 668b4616
test ax , ax ;0x004ae9ed : 6685c0
lea edi , dword ptr [ esi + 16 ] ;0x004ae9f0 : 8d7e10
je Label477a2f ;0x004ae9f3 : 0f843690fcff
test ah , ah ;0x004ae9f9 : 84e4
js Label591d82 ;0x004ae9fb : 0f8881330e00
mov eax , 01h ;0x004aea01 : b801000000
Label4aea06 ::
xor ebx , ebx ;0x004aea06 : 33db
test eax , eax ;0x004aea08 : 85c0
mov eax , dword ptr [ esp + 24 ] ;0x004aea0a : 8b442418
setl bl ;0x004aea0e : 0f9cc3
test eax , eax ;0x004aea11 : 85c0
jne Label4aea1c ;0x004aea13 : 7507
mov ecx , edi ;0x004aea15 : 8bcf
call Fun53e46a ;0x004aea17 : e84efa0800
Label4aea1c ::
test ebx , ebx ;0x004aea1c : 85db
fld qword ptr [edi] ;0x004aea1e : dd07
jne Label591d9f ;0x004aea20 : 0f8579330e00
Label4aea26 ::
mov eax , dword ptr [ esp + 20 ] ;0x004aea26 : 8b442414
fld qword ptr [ 8 * ebp + offset Data922520 ] ;0x004aea2a : dd04ed20259200
sub esp , 08h ;0x004aea31 : 83ec08
fstp qword ptr [ esp + 40 ] ;0x004aea34 : dd5c2428
test eax , eax ;0x004aea38 : 85c0
jne Label591da6 ;0x004aea3a : 0f8566330e00
fld qword ptr [ esp + 40 ] ;0x004aea40 : dd442428
fmul st(0) , st(1) ;0x004aea44 : d8c9
;
; ---------------------------------------------------------------------------------------------------
; VFP 9.0 FIX - ADJUST THE PRECISION OF THE FLOATING-POINT NUMBERS FOR THE ROUND() FUNCTION
; February 2017
; ---------------------------------------------------------------------------------------------------
; CCB
;
; Sometimes the ROUND() function returns wrong result, for example,
; ?ROUND(512.9250000000,2) && Displays 512.92
; ?ROUND(512.925000000,2) && Displays 512.93
;
; 2017/2/26, by ccb
;
cmp dword ptr vfpa_sys9001_data,00h
je Label4aea46
fld st(0)
fabs
fldlg2
fxch st(1)
fyl2x
fld st(0)
fistp dword ptr [ esp ]
mov ecx,dword ptr [ esp ]
cmp ecx,80000000h
je Label4aea45
and ecx,80000000h
test ecx,ecx
jne Label4aea45
mov eax,0
IFDEF RAX
push esi
lea esi , dword ptr [ Data922438 ]
fadd qword ptr [ 8 * eax + esi ]
pop esi
ELSE
fadd qword ptr [ 8 * eax + offset Data922438 ]
ENDIF
mov eax,0Ah
IFDEF RAX
push esi
lea esi , dword ptr [ Data922438 ]
fadd qword ptr [ 8 * eax + esi ]
pop esi
ELSE
fadd qword ptr [ 8 * eax + offset Data922438 ]
ENDIF
fistp dword ptr [ esp ]
mov eax,0Fh
sub eax,dword ptr [ esp ]
cmp eax,0
jl Label4aea46
cmp eax,0Fh
jg Label4aea46
fld st(0)
fistp dword ptr [ esp ]
mov ecx,dword ptr [ esp ]
cmp ecx,80000000h
je Label4aea46
and ecx,80000000h
test ecx,ecx
jne Label4aea44
IFDEF RAX
push esi
lea esi , dword ptr [ Data922438 ]
fadd qword ptr [ 8 * eax + esi ]
pop esi
ELSE
fadd qword ptr [ 8 * eax + offset Data922438 ]
ENDIF
jmp Label4aea46
Label4aea44 ::
IFDEF RAX
push esi
lea esi , dword ptr [ Data922438 ]
fsub qword ptr [ 8 * eax + esi ]
pop esi
ELSE
fsub qword ptr [ 8 * eax + offset Data922438 ]
ENDIF
jmp Label4aea46
Label4aea45 ::
fstp st(0)
jmp Label4aea46
Label4aea46 ::
fadd qword ptr [ Data9256b0 ] ;0x004aea46 : dc05b0569200
fstp qword ptr [ esp ] ;0x004aea4c : dd1c24
fstp st(0) ;0x004aea4f : ddd8
call floor ;0x004aea51 : ff15a0799100
fdiv qword ptr [ esp + 40 ] ;0x004aea57 : dc742428
Label4aea5b ::
add esp , 08h ;0x004aea5b : 83c408
test ebx , ebx ;0x004aea5e : 85db
jne Label591dc2 ;0x004aea60 : 0f855c330e00
Label4aea66 ::
mov eax , dword ptr [ esp + 24 ] ;0x004aea66 : 8b442418
fstp qword ptr [edi] ;0x004aea6a : dd1f
test eax , eax ;0x004aea6c : 85c0
jne Label591dc9 ;0x004aea6e : 0f8555330e00
mov eax , dword ptr [ esp + 20 ] ;0x004aea74 : 8b442414
test eax , eax ;0x004aea78 : 85c0
jne Label591e1c ;0x004aea7a : 0f859c330e00
mov eax , dword ptr [ esi + 8 ] ;0x004aea80 : 8b4608
cmp ebp , eax ;0x004aea83 : 3be8
ja Label591e28 ;0x004aea85 : 0f879d330e00
Label4aea8b ::
cmp ebp , 012h ;0x004aea8b : 83fd12
jnl Label591e42 ;0x004aea8e : 0f8dae330e00
Label4aea94 ::
mov dword ptr [ esi + 8 ] , ebp ;0x004aea94 : 896e08
Label4aea97 ::
mov eax , dword ptr [ esi + 4 ] ;0x004aea97 : 8b4604
cmp eax , 028h ;0x004aea9a : 83f828
jnl Label4aeaa3 ;0x004aea9d : 7d04
inc eax ;0x004aea9f : 40
mov dword ptr [ esi + 4 ] , eax ;0x004aeaa0 : 894604
Label4aeaa3 ::
mov eax , dword ptr [ esp + 28 ] ;0x004aeaa3 : 8b44241c
test eax , eax ;0x004aeaa7 : 85c0
jne Label591e4c ;0x004aeaa9 : 0f859d330e00
Label4aeaaf ::
pop edi ;0x004aeaaf : 5f
pop esi ;0x004aeab0 : 5e
pop ebp ;0x004aeab1 : 5d
pop ebx ;0x004aeab2 : 5b
mov esp , ebp ;0x004aeab3 : 8be5
pop ebp ;0x004aeab5 : 5d
ret ;0x004aeab6 : c3
4. APPLIES TO:
VFP 6.0.8167.0
VFP 6.0.8961.0 (SP5)
VFP 7.0.0.9262
VFP 7.0.0.9465 (SP1)
VFP 8.0.0.2521
VFP 8.0.0.3117 (SP1)
VFP 9.0.0.2412
VFP 9.0.0.3504 (SP1)
VFP 9.0.0.4611 (SP2)
VFP 9.0.0.5015 (SP2)
VFP 9.0.0.5411 (SP2)
VFP 9.0.0.5721 (SP2)
VFP 9.0.0.5815 (SP2)
VFP 9.0.0.6303 (SP2)
VFP 9.0.0.6602 (SP2)
VFP 9.0.0.7423 (SP2)
The bug has been fixed in VFP Advanced.
5. REFERENCE WEBSITES:
1, baiyujia.com:
http://www.baiyujia.com
http://www.baiyujia.com/vfpdocuments/f_vfp9fix23.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix37.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix38.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix97.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix98.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix123.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix124.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix255.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix256.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix257.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix258.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix259.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix260.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix303.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix304.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix305.asp
http://www.baiyujia.com/vfpdocuments/f_vfp9fix306.asp
2, microsoft.com:
https://social.msdn.microsoft.com/Forums/en-US/cee551be-35f8-4ddc-a07e-51d2d7f79345/rounding-bug-in-vfp8
https://social.msdn.microsoft.com/Forums/en-US/a8725753-5815-4d45-aa7b-bdead9277986/how-can-i-subract-datetime-values
3, foxite.com:
https://www.foxite.com/archives/round-problem-0000070917.htm
https://www.foxite.com/archives/interesting-behavior-of-inputmask-0000406050.htm
https://www.foxite.com/archives/round-0000223089.htm
https://www.foxite.com/archives/how-to-compute-the-elapsed-time-in-foxpro-0000058524.htm
https://www.foxite.com/archives/int-strange-behavior-0000445268.htm
4, tek-tips.com:
http://www.tek-tips.com/viewthread.cfm?qid=531793
http://www.tek-tips.com/viewthread.cfm?qid=1738764
5, foxpert.com:
http://www.foxpert.com/foxpro/knowlbits/files/knowlbits_201102_1.html
6, fox-id.org:
http://www.fox-id.org/smf/general-code-activex-com/hasil-perhitungan-vfp-kok-beda-sama-kalkulator/
6. OTHER:
For reference only, there is no guarantees.
Any questions or suggestions, please send me an email at ccb2000@163.com.
|