[fpc-pascal] Currency and ABS(-674.59)

[Zaher Dirkey]

I found this problem in Delphi and FPC, please test it and confirm if it is
a bug.

-------------------
program test_abs_curr;
uses
sysutils;

var
a: Currency;
begin
a := 674.59;
if a<>abs(-a) then
writeln('not equal')
else
writeln('equal');

end.
-------------------

I am using last reversion of Lazarus, FPC 2.6

Best Regards
Zaher Dirkey

[Jeppe Græsdal Johansen]

Post by Zaher Dirkey
I found this problem in Delphi and FPC, please test it and confirm if
it is a bug.
-------------------
program test_abs_curr;
uses
sysutils;
var
a: Currency;
begin
a := 674.59;
if a<>abs(-a) then
writeln('not equal')
else
writeln('equal');
end.
-------------------
I am using last reversion of Lazarus, FPC 2.6
Best Regards
Zaher Dirkey

I get "not equal" for i386-win32, however with the following program I
get "equal"

program test_abs_curr;
uses
sysutils;

var
a,b: currency;
begin
a := 674.59;
b := abs(-a);
if a<>b then
writeln('not equal')
else
writeln('equal');

end.

[Jonas Maebe]

Post by Zaher Dirkey
I found this problem in Delphi and FPC, please test it and confirm if it is
a bug.

The problem is caused by the facts that
a) on i386, currency operations are calculated using the 80x87 floating point unit
b) there is no separate abs() for currency, hence abs(currency) becomes currency(abs(extended(currency)))

Conversion from currency to a floating point type means dividing by 10000 and converting back means multiplying again by 10000. The division by 10000 results in a rounding error, which in turn results in the unexpected comparison result (the result of the abs(currency) in your expression is not stored to memory, and hence still contains the full 80 bits of "precision" from the division/multiplication combo). That is why Jeppe's program works correctly: then the result is stored back into a variable, which hides the rounding error.

It's basically a property of the fact that on i386, currency is a floating point type handled by the fpu that emulates a fixed point type. This particular problem could obviously be resolved by adding a currency-specific version of abs(), but in general the problem can occur with several other expressions too (as soon as for one reason or another, the operation is not implemented specifically for currency and the compiler inserts a conversion to another floating point type).


Jonas

[Zaher Dirkey]

Post by Jonas Maebe
This particular problem could obviously be resolved by adding a
currency-specific version of abs()

Can i ask to add overload function for ABS(Currency) be a feature request
in FPC?

Thanks

I am using last reversion of Lazarus, FPC 2.6

Best Regards
Zaher Dirkey

[m***@wisa.be]

Post by Zaher Dirkey

Post by Jonas Maebe
This particular problem could obviously be resolved by adding a
currency-specific version of abs()

Can i ask to add overload function for ABS(Currency) be a feature request
in FPC?

Yes, please add an entry in the bugtracker.

Michael.

[Jonas Maebe]

Post by m***@wisa.be

Post by Zaher Dirkey

Post by Jonas Maebe
This particular problem could obviously be resolved by adding a
currency-specific version of abs()

Can i ask to add overload function for ABS(Currency) be a feature request
in FPC?

Yes, please add an entry in the bugtracker.

Note that this requires compiler patching, because abs() is internal
(it can be used in constant expressions). I would personally argue to
do away entirely with the "treat currency as a floating point type on
i386 so it can use the 80x87", and instead map it implementation-wise
to int64 like on all other platforms. You may lose a bit of
performance, but you'll gain consistency. And you won't need hacks
like this (which, as mentioned before, only solves one particular use-
case, and so I'm not very much in favour of doing this).


Jonas

[m***@wisa.be]

Post by m***@wisa.be

On Mon, Mar 5, 2012 at 12:22 AM, Jonas Maebe

Post by Jonas Maebe
This particular problem could obviously be resolved by adding a
currency-specific version of abs()

Can i ask to add overload function for ABS(Currency) be a feature request
in FPC?

Yes, please add an entry in the bugtracker.

Note that this requires compiler patching, because abs() is internal (it can
be used in constant expressions). I would personally argue to do away
entirely with the "treat currency as a floating point type on i386 so it can
use the 80x87", and instead map it implementation-wise to int64 like on all
other platforms. You may lose a bit of performance, but you'll gain
consistency. And you won't need hacks like this (which, as mentioned before,
only solves one particular use-case, and so I'm not very much in favour of
doing this).

As far as I know, Currency is always a scaled int64, and didn't interpret the
request as a request to change that. I missed probably part of the
argumentation but on the face of it, having a ABS(Currency) seems like a
reasonable request.

Michael.

[Sven Barth]

Post by m***@wisa.be

Post by Jonas Maebe

Post by m***@wisa.be

On Mon, Mar 5, 2012 at 12:22 AM, Jonas Maebe

Post by Jonas Maebe
This particular problem could obviously be resolved by adding a
currency-specific version of abs()

Can i ask to add overload function for ABS(Currency) be a feature request
in FPC?

Yes, please add an entry in the bugtracker.

Note that this requires compiler patching, because abs() is internal
(it can be used in constant expressions). I would personally argue to
do away entirely with the "treat currency as a floating point type on
i386 so it can use the 80x87", and instead map it implementation-wise
to int64 like on all other platforms. You may lose a bit of
performance, but you'll gain consistency. And you won't need hacks
like this (which, as mentioned before, only solves one particular
use-case, and so I'm not very much in favour of doing this).

As far as I know, Currency is always a scaled int64, and didn't interpret the
request as a request to change that. I missed probably part of the
argumentation but on the face of it, having a ABS(Currency) seems like a
reasonable request.

No, Currency is based on Extended on i386 and x86_64 (except win64!).

Regards,
Sven

[Zaher Dirkey]

Post by Sven Barth
No, Currency is based on Extended on i386 and x86_64 (except win64!).

Hmm, but it is break the balance in the budget of my customer.

I resolved it by adding CurrABS function, but i don't think it is good idea.

Best Regards
Zaher Dirkey

[Jonas Maebe]

Post by m***@wisa.be
As far as I know, Currency is always a scaled int64, and didn't interpret the
request as a request to change that.

The problem is that on i386 (and in Delphi on i386), operations on the
currency type are handled using the fpu. The result is that until
truncation (which only happens when storing the value to memory), you
can have rounding errors in the intermediate results. The fpu on other
platforms has not enough precision to exactly represent the entire
int64 range, so there we implement the currency type using an actual
int64 and with int64 arithmetic instead. Hence you don't get rounding
errors in intermediate results there.

Post by m***@wisa.be
I missed probably part of the
argumentation but on the face of it, having a ABS(Currency) seems like a
reasonable request.

And also division for currency then, I guess (interestingly, the
program below also gives an error on non-x86 and win64 platforms
currently, but that's due to a code generation bug rather than a
rounding error -- the result from the second calculation is currently
completely bogus there):

var
c: currency;
begin
c:=(high(int64) div 10000) + 0.0333;
writeln(c/10.0);
writeln(c*(1/10.0));
if (c/10.0) <> c*(1/10.0) then
writeln('error');
end.

Output on Linux/i386:

$ ./curr2
9.223372036854770330E+13
9.223372036854770330E+13
error

Kylix also gives an error here, although it prints lets digits for the
writeln.

Anyway, my point is: a currency type implemented using floating point
math is basically a contradiction in terms, and no amount of duct tape
can fix that (except for storing the result to memory after every
single calculation, which would kill pretty much any speed advantage
that using the FPU could offer). So I'd prefer not to start adding
duct tape, since that's "een straatje zonder einde" :)


Jonas

[Flávio Etrusco]

Post by Jonas Maebe

Post by m***@wisa.be
As far as I know, Currency is always a scaled int64, and didn't interpret the
request as a request to change that.

The problem is that on i386 (and in Delphi on i386), operations on the
currency type are handled using the fpu.

(...)

This comes as a big surprise to me. I always thought the docs
definition (http://docwiki.embarcadero.com/RADStudio/en/Internal_Data_Formats#The_Currency_type)
implied the FPU wasn't used.
I wonder how that didn't turn out on the rare times I used the
Currency type. I'm glad I never worked with banking/financial systems
:-$

-Flávio

[Zaher Dirkey]

Post by Flávio Etrusco
I'm glad I never worked with banking/financial systems
:-$

In accounting/financial systems there is Debit and Credit numbers it must
equal and Debit - Credit = 0, with this bug it fail.
I am an accountant and i know if there is 0.0001 diff in balance sheet, my
boss will fire me :P

I still believe the results in i386 must be equal also to other
systems/platforms even if it calculated in slow code.

Best Regards
Zaher Dirkey

[Graeme Geldenhuys]

Post by Zaher Dirkey
In accounting/financial systems there is Debit and Credit numbers it must

equal and Debit - Credit = 0, with this bug it fail.

Post by Zaher Dirkey
I am an accountant and i know if there is 0.0001 diff in balance sheet,

my boss will fire me :P

This is why our accounting app uses Integer math (and when stored in the
database). Only when amounts are displayed to the user, do we convert the
integer value to currency. This greatly reduced our calculation and
rounding problems.

Graeme.

[Marcos Douglas]

On Wed, Mar 7, 2012 at 7:43 PM, Graeme Geldenhuys

Post by Graeme Geldenhuys

Post by Zaher Dirkey
In accounting/financial systems there is Debit and Credit numbers it must
equal and Debit - Credit = 0, with this bug it fail.
I am an accountant and i know if there is 0.0001 diff in balance sheet, my
boss will fire me :P

This is why our accounting app uses Integer math (and when stored in the
database). Only when amounts are displayed to the user, do we convert the
integer value to currency. This greatly reduced our calculation and rounding
problems.

Integer math... what did you mean?

Marcos Douglas

[Graeme Geldenhuys]

Post by Marcos Douglas
Integer math... what did you mean?

Over the years we have experience many problems with currency,
floating point and date/time calculations and storage. So we tweaked
our software to handle such data slightly different.

If a user enters and amount of 1500.25, it is immediately converted to
a Integer as 150025. We have a global project wide setting that set
the amount of decimals to support - default being 2. So then all
calculations and comparisons (which is a big issue in floating point
values) are done with Integer types. It is stored in the database as
an Integer as well. But when it is displayed anywhere to the user, it
is reformatted with decimals (for the display only).

Similar thing with dates. We always store it as a string using the ISO
8601 [ http://www.cl.cam.ac.uk/~mgk25/iso-time.html ] format. This
way, no matter the database being used (MS SQL Server, Firebird,
Oracle etc.) with their various timestamp field types, or what locale
the database server used for that specific database (which affects the
timestamp field type storage format), we always know the format the
date and time is stored in. When the data is loaded we have a
ISO_to_DateTime() routine (and the reverse when stored) that converts
it to a TDateTime in memory. Displaying of the date/time to the end
user can be configure by the end-user (or application or system wide).
Because the date/time is always in yyyymmddThhmmss format, even SQL
working directly with the date/time fields can still do comparisons
like is one date larger that the other.

Since we have used these two methods (for the past 5 years) in our
projects, we have just about eliminated all issues we previous had
with currency, float and timestamp data.


And as was mentioned, rounding and comparisons are major issues with
floating point data, and non-existent in Integer data.

[Marcos Douglas]

On Thu, Mar 8, 2012 at 4:41 AM, Graeme Geldenhuys

Post by Graeme Geldenhuys

Post by Marcos Douglas
Integer math... what did you mean?

Over the years we have experience many problems with currency,
floating point and date/time calculations and storage. So we tweaked
our software to handle such data slightly different.
If a user enters and amount of 1500.25, it is immediately converted to
a Integer as 150025. We have a global project wide setting that set
the amount of decimals to support - default being 2. So then all
calculations and comparisons (which is a big issue in floating point
values) are done with Integer types. It is stored in the database as
an Integer as well. But when it is displayed anywhere to the user, it
is reformatted with decimals (for the display only).
Similar thing with dates. We always store it as a string using the ISO
8601  [ http://www.cl.cam.ac.uk/~mgk25/iso-time.html ] format. This
way, no matter the database being used (MS SQL Server, Firebird,
Oracle etc.) with their various timestamp field types, or what locale
the database server used for that specific database (which affects the
timestamp field type storage format), we always know the format the
date and time is stored in. When the data is loaded we have a
ISO_to_DateTime() routine (and the reverse when stored) that converts
it to a TDateTime in memory. Displaying of the date/time to the end
user can be configure by the end-user (or application or system wide).
Because the date/time is always in yyyymmddThhmmss format, even SQL
working directly with the date/time fields can still do comparisons
like is one date larger that the other.
Since we have used these two methods (for the past 5 years) in our
projects, we have just about eliminated all issues we previous had
with currency, float and timestamp data.
And as was mentioned, rounding and comparisons are major issues with
floating point data, and non-existent in Integer data.

So, if you have two values to storage (amount of decimals is 4) e.g.
100.34524 and 2,000.2 you do:
100.34524 = 1003452
2,000.2 = 20002000

I never heard about this technique before. Very good!

For DateTime I already do this, but not in all cases.

Marcos Douglas

[Graeme Geldenhuys]

Post by Marcos Douglas
So, if you have two values to storage (amount of decimals is 4) e.g.
100.34524 = 1003452
2,000.2 = 20002000

Correct.

We then implemented our own function [eg: M2AmountToCurr() ] which
formats such a number [only when it needs to be displayed] into
something the end-user would understand, and using whatever formatting
style (based on locale or defined by system admin) and defined decimal
count. We also have our own rounding function [when converting
floating point to integer] - adhering to the specifications defined by
our finance department.

Post by Marcos Douglas
I never heard about this technique before. Very good!

Yes, it works very well for us.

[Sven Barth]

Post by Marcos Douglas
So, if you have two values to storage (amount of decimals is 4) e.g.
100.34524 = 1003452
2,000.2 = 20002000
I never heard about this technique before. Very good!

The buzzword for this is Fixed Point Arithmetic. See here:
http://en.wikipedia.org/wiki/Fixed-point_arithmetic

Regards,
Sven

[Jonas Maebe]

The buzzword for this is Fixed Point Arithmetic. See here: http://en.wikipedia.org/wiki/Fixed-point_arithmetic

And in principle, it's exactly what currency should use. And in fact,
it is what it does use in FPC on platforms that don't support an x87
fpu.


Jonas

[Joao Morais]

Post by Sven Barth
The buzzword for this is Fixed Point Arithmetic. See
here: http://en.wikipedia.org/wiki/Fixed-point_arithmetic

And in principle, it's exactly what currency should use. And in fact, it is
what it does use in FPC on platforms that don't support an x87 fpu.

What about promote the consistency between all platforms and implement
currency as a scaled integer? After all, internally, it isn't a
floating point format.

Joao Morais

[Zaher Dirkey]

Post by Marcos Douglas
Integer math... what did you mean?

Integer math = Currency or in DB numeric(18, 4)

in Firebird
create domain AMOUNT as numeric(18,4);

On Thu, Mar 8, 2012 at 12:43 AM, Graeme Geldenhuys

Post by Marcos Douglas
This is why our accounting app uses Integer math (and when stored in the
database). Only when amounts are displayed to the user, do we convert the
integer value to currency. This greatly reduced our calculation and
rounding problems.

Yes i do that from more than 10 years, Float/Extended have problem when
rounding it and sum that rounding, it is only good in Science not
Accounting.

Currency for me is Int64 (* int of 10000 or binary shifted) it is more
stable and not lose any cent when sum the balances, until i passed it to
ABS, ABS convert it to float and return back to Currency, that my problem
here.

In the future i like to have new kind of Currency (integer) but more extend
(* 1000000 for example).

Best Regards
Zaher Dirkey

[Ludo Brands]

Post by Jonas Maebe

Post by m***@wisa.be

Post by Jonas Maebe
Note that this requires compiler patching, because abs()

is internal

Post by m***@wisa.be

Post by Jonas Maebe
(it can be used in constant expressions). I would

personally argue to

Post by m***@wisa.be

Post by Jonas Maebe
do away entirely with the "treat currency as a floating

point type on

Post by m***@wisa.be

Post by Jonas Maebe
i386 so it can use the 80x87", and instead map it

implementation-wise

Post by m***@wisa.be

Post by Jonas Maebe
to int64 like on all other platforms. You may lose a bit of
performance, but you'll gain consistency. And you won't need hacks
like this (which, as mentioned before, only solves one particular
use-case, and so I'm not very much in favour of doing this).

As far as I know, Currency is always a scaled int64, and didn't
interpret the request as a request to change that. I missed

probably

Post by m***@wisa.be
part of the argumentation but on the face of it, having a
ABS(Currency) seems like a reasonable request.

No, Currency is based on Extended on i386 and x86_64 (except win64!).
Regards,
Sven

There is also a lot of code in the rtl and fcl-db that supposes currency =
double. Fe.

Function CurrToStr(Value: Currency; Const FormatSettings: TFormatSettings):
string;
begin
Result:=FloatToStrF(Value,ffGeneral,-1,0,FormatSettings);
end;

Or

TCurrencyField = class(TFloatField)

Ludo

[Sven Barth]

Post by Ludo Brands

Post by Jonas Maebe

Post by m***@wisa.be

Post by Jonas Maebe
Note that this requires compiler patching, because abs()

is internal

Post by m***@wisa.be

Post by Jonas Maebe
(it can be used in constant expressions). I would

personally argue to

Post by m***@wisa.be

Post by Jonas Maebe
do away entirely with the "treat currency as a floating

point type on

Post by m***@wisa.be

Post by Jonas Maebe
i386 so it can use the 80x87", and instead map it

implementation-wise

Post by m***@wisa.be

Post by Jonas Maebe
to int64 like on all other platforms. You may lose a bit of
performance, but you'll gain consistency. And you won't need hacks
like this (which, as mentioned before, only solves one particular
use-case, and so I'm not very much in favour of doing this).

As far as I know, Currency is always a scaled int64, and didn't
interpret the request as a request to change that. I missed

probably

Post by m***@wisa.be
part of the argumentation but on the face of it, having a
ABS(Currency) seems like a reasonable request.

No, Currency is based on Extended on i386 and x86_64 (except win64!).
Regards,
Sven

There is also a lot of code in the rtl and fcl-db that supposes currency =
double. Fe.
string;
begin
Result:=FloatToStrF(Value,ffGeneral,-1,0,FormatSettings);
end;
Or
TCurrencyField = class(TFloatField)

This should not be a problem as such code already works on non-i386
platforms where currency is a Int64 ;)

Regards,
Sven

[Jonas Maebe]

Post by Ludo Brands
There is also a lot of code in the rtl and fcl-db that supposes currency =
double. Fe.
string;
begin
Result:=FloatToStrF(Value,ffGeneral,-1,0,FormatSettings);
end;

FloatToStrF() is overloaded for currency. It's normal that currency<-

Post by Ludo Brands
string conversions are overloaded, since it's a separate type with

separate formatting rules/restrictions. That is unrelated to the
internal representation.


Jonas

[Martin Schreiber]

Post by Ludo Brands

Post by Sven Barth
No, Currency is based on Extended on i386 and x86_64 (except win64!).

There is also a lot of code in the rtl and fcl-db that supposes currency =
double. Fe.
string;
begin
Result:=FloatToStrF(Value,ffGeneral,-1,0,FormatSettings);
end;
Or
TCurrencyField = class(TFloatField)

TCurrencyField datatype is double. The difference to TFloatField is
the "GetText" formatting where currency formatting of the current locale will
be used instead of float formatting AFAIK. The TField class with currency
datatype is TBCDField.

Martin

[Ludo Brands]

Post by Sven Barth

Post by Ludo Brands
There is also a lot of code in the rtl and fcl-db that supposes
currency = double. Fe.
TFormatSettings): string; begin
Result:=FloatToStrF(Value,ffGeneral,-1,0,FormatSettings);
end;
Or
TCurrencyField = class(TFloatField)

This should not be a problem as such code already works on non-i386
platforms where currency is a Int64 ;)
Regards,
Sven

If you don't consider all the useless type conversions being a problem ;)

When all the platforms use a scaled Int64 (which I'm in favor of) you can't
leave such code in place. This thread started with a rounding problem caused
by such type conversions.

Ludo