#1
|
|||
|
|||
Divide number bigger than a long
If I do integer divide on a number bigger than a long (32 bits), I get an overflow error. Is there a way to declare a number as a double? If not, how do you do arithmetic on a number bigger than a long (32 bits)?
Example - need to divide by 2^14 to strip off lower 10 bits to get the value in bits 11-34: inputNum = 14417936436 newNum = inputNum \ 16384 <---- Overflow error |
#2
|
|||
|
|||
Hi AndrewEK,
This is more of a scripting *language* question. This particular scripting forum is specifically set up for discussion of scripting techniques that are applicable to VanDyke Software products. Because your question is not VanDyke Software specific, you might consider posting the same question to a forum more generally focused on the scripting language you are using.
__________________
Thanks, --Brenda VanDyke Software Technical Support support@vandyke.com (505) 332-5730 |
#3
|
|||
|
|||
Hi AndrewEK,
But you might try / not \. ![]()
__________________
Thanks, --Brenda VanDyke Software Technical Support support@vandyke.com (505) 332-5730 |
#4
|
|||
|
|||
As a point of reference, on v8.5.3 (64-bit version) I got 880001 with no overflow error.
|
![]() |
Tags |
divide , overflow |
Thread Tools | |
Display Modes | Rate This Thread |
|
|