On Tue, Dec 1, 2009 at 17:31, Ale Muñoz bomberstudios@gmail.com wrote:

I expected something akin to what every decent tool out there produces:

$ ruby -e "puts (72/2.54)*2.54" 72.0

$ php -r "echo((72/2.54)*2.54);" 72

$ perl -e "print ((72/2.54)*2.54)" 72

heck, even JavaScript gets it right...

The fact that the reduced precision those tools use masks their use of imprecise numbers results in an error which matches your expectations doesn’t actually make them decent.

> ruby -e 'puts 72/2.54' 28.3464566929134

> perl -e "print 72/2.54" 28.3464566929134%

> php -r "echo((72/2.54);" 28.346456692913%

Contrast with the tool TextMate uses underneath:

> bc -l <<<'72 / 2.54' 28.34645669291338582677

Funnily enough, the "Send Line to Google Calculator" on the same bundle gets it right...

…as demonstrated in this expression:

72 / 2.54 = 28.3464567

I’m afraid I don’t think I understand your criteria of either decency /or/ correctness… Have you heard of “What Every Computer Scientist Should Know About Floating-Point Arithmetic”[1]? Maybe it could help establish some common ground here…

[1] http://docs.sun.com/source/806-3568/ncg_goldberg.html

On 1 Dec 2009, at 10:54, Timothy Bates wrote:

What else did you expect? It's using a calculator not a symbolic math package, so 71.99999999999999999999 is spot on.

That was my initial reaction, too. It's just normal floating-point imprecision. But I tried it in C with single-precision floats, and got 72.0 as a result. Hmmm...

Okay, so what's the bundle doing? Here's the code:

printf " = %s" `{ tr -d ,; echo; } | bc -l`| perl -pe 's/(.[^0]+)0+$|.0+$/$1/'

Yikes! Good lord, that's ugly! The culprit is the 'bc' command, which is doing the actual calculation. All the rest is just for formatting. Getting rid of all the rest of the cruft,

macduff:/tmp$ echo '(72 / 2.54) * 2.54' | bc -l 71.99999999999999999999

You could take it up with the Free Software Foundation, though I suspect they have a fanatical academic reason why they're doing it the One True Way.

To be fair, I don't think bc is the right tool for this job anyway. It uses an arbitrary-precision library rather than the floating-point that everything else uses. Both methods have their advantages and disadvantages, but for the precision most people tend to deal with I think the standard floating-point libraries do a better job. At the very least, we're more accustomed to dealing with their idiosyncrasies.

Personally, I'd re-write the bundle to avoid bc. Since perl is being used anyway it wouldn't be hard to write the command such that perl does the whole thing, eliminating that ugly pipeline altogether.