VanDyke Software Forums

VanDyke Software Forums (https://forums.vandyke.com/index.php)
-   Scripting (https://forums.vandyke.com/forumdisplay.php?f=14)
-   -   Divide number bigger than a long (https://forums.vandyke.com/showthread.php?t=13859)

AndrewEK 08-02-2019 02:58 PM

Divide number bigger than a long
 
If I do integer divide on a number bigger than a long (32 bits), I get an overflow error. Is there a way to declare a number as a double? If not, how do you do arithmetic on a number bigger than a long (32 bits)?

Example - need to divide by 2^14 to strip off lower 10 bits to get
the value in bits 11-34:

inputNum = 14417936436
newNum = inputNum \ 16384 <---- Overflow error

bgagnon 08-02-2019 03:14 PM

Hi AndrewEK,

This is more of a scripting *language* question.

This particular scripting forum is specifically set up for discussion of scripting techniques that are applicable to VanDyke Software products.

Because your question is not VanDyke Software specific, you might consider posting the same question to a forum more generally focused on the scripting language you are using.

bgagnon 08-02-2019 03:48 PM

Hi AndrewEK,

But you might try / not \. :)

gregg 08-27-2019 09:50 AM

As a point of reference, on v8.5.3 (64-bit version) I got 880001 with no overflow error.


All times are GMT -6. The time now is 05:23 PM.