Hello Michael A. Lowry,
well i think that is an issue depending on how floating point numbers are processed in the automation engine. The number ist not stored in memory as 3.1459. Mostly near all programming languages store it binary as a sequence of 0 and 1. There are a finite number of doubles, so only a specific subset of the real numbers are the exact value of a double. When you create a double literal, the decimal number you type is represented by the nearest of those values. When you output a double, by default, it is shown as the shortest decimal fraction that would round to it on input.
That is the theoretical explanation to it. But how to handle it. With Automation Engine i would use Format to clarify the output.
From the view of an developer, there might be some techiques to prevent this behavior. In Java there is a class called BigDecimal to handle this problem. Maybe a new ideation can be a solution to handle the problem, or you can try to see it as an error in representation float numbers an open a case.
Best Regards
Andrzej Golaszewski
Original Message:
Sent: Feb 16, 2024 05:57 AM
From: Michael A. Lowry
Subject: Float precision problem
I can understand that errors can creep in during calculations. But in the example I provided above, the script is simply assigning a value. Is the Automation Engine incapable of representing the number 3.1459?