[Scilab-users] lcm() output type: how to avoid overflowing and get an actual result?

[Tim Wescott]

So, a quick test gives me:

-->A = uint8([123 423]); A / lcm(A)
 ans  =
 
  2  2  

This looks pretty "least common multiple" to me.  If I want a least
common multiple over decimal numbers or floats or bigger integers or
whatever, then it's my responsibility to tell the function that's what
I want by appropriately casting the type of the input.

A note in the help section about this (desirable, IMHO) behavior would
probably be a Good Idea though.

On Wed, 2017-03-15 at 17:04 +0100, Samuel Gougeon wrote:
> Hello,
> 
> Processing the bug http://bugzilla.scilab.org/15017 for the lcm()
> function
> deserves an open discussion about the datatype of the result when the
> operand
> is a matrix of encoded integers.
> Presently, the result of lcm(A) has the type of A:
> 
> --> r = lcm(int16([123 423])), typeof(r)
>  r  = 
>   17343
>  ans  =
>  int16
> 
> --> r = lcm(uint8([123 423])), typeof(r)
>  r  = 
>   61
>  ans  =
>  uint8
> 
> This behavior is questionable, in the way that the Least Common
> Multiple of some
> integers is always equal or most often greater than the maximum of
> the operand.
> Therefore, when the operand is a matrix of encoded integers, the LCM
> can easily
> (and most often) yield an overflow, and so yield a wrapped result,
> being most often
> unrelevant and misleading:
> Example:
> --> A = int8([2 3 5 7]);
> --> lcm(A)
>  ans  =
>  -46        // 210 expected
> --> typeof(lcm(A))
>  ans  =
>  int8
> 
> So we see that even with few and small input values, as expected the
> result is already
> "corrupted".
> 
> This bad effect does not happens with gcd() because its results is
> always
> smaller or equal to the maximum of the operand components.
> 
> Octave behaves in the same bad way than Scilab, just saturating to
> the highest
> integer value instead of wrapping it, but in both cases the result is
> wrong:
> 
> >> A = int8([2 3 5 7]);
> >> lcm(A(1), A(2), A(3), A(4))
> ans = 127
> 
> So, the question is: Do we improve the situation, and if we do, in
> which way?
> Here are some suggestions:
> 
> * The result is always promoted to the next class of inttype:
>   int8 => int16, uint8 => uint16, int16 => int32, etc
>   int64 and uint64 could be promoted into decimal numbers, but with a
> possible
>   loss of accuracy (but beyond 2^63)
> 
> * The result is always promoted to the most accurate (int64 or)
> uint64 class
> 
> * The result is always promoted to the decimal (aka floating point)
> class
> 
> My (slight) preference would be to cast the result always into the
> decimal class. Why?
>  - the type of the result would be always the same: decimal
> 
>  - it would be OK and lossless for all int8, uint8, int16, and uint16
> operands,
>    and for all int32, uint32, int64, uint64 smaller than 2^26
>    OK, we could loose some low bits when processing some big 32 or 64
> bits integers,
>    but at least we would have a result ; otherwise, casting to uint64
> could still
>    lead to some overflow, so no actual result at all. 
> 
>  - Usual +,-,*,/,^ operators between an int# or uint# and a decimal
> number forces
>    the result to the encoded integer type. So this choice would let
> any further
>    operations recasting the lcm() result into the original inttype
> (instead of 
>    promoting all further results in an unexpected way).
> 
> What's your opinion? What would you prefer working with?
> 
> Hope reading you soon
> Samuel
> 
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users
-- 

Tim Wescott
www.wescottdesign.com
Control & Communications systems, circuit & software design.
Phone: 503.631.7815
Cell:  503.349.8432