[scilab-Users] RE: Concatenation with hypermatrix

[Berns Buenaobra]

Hello all:

This is the best!

Many thanks Michaël Baudin!

Regards,

Berns B.


On Thu, Jun 9, 2011 at 4:09 PM, Michaël Baudin <michael.baudin at scilab.org>wrote:

> Hi,
>
> Do you have a simple script to reproduce your performance issue ?
>
> You might be interested by "Programming in Scilab", especially the section
> "3.3 Hypermatrices" and the section "3.12 Comparison of data types" :
>
> http://forge.scilab.org/index.php/p/docprogscilab/downloads/
>
> In the Figure 19, I analyzed the current implementation of hypermatrices in
> Scilab v5. Here is one explanation for the potential lack of performances,
> associated with the current implementation. Hypermatrices are partially
> implemented with mlists, supported by both compiled source code and Scilab
> macros. In v5, the hypermatrix is internally stored as a n-by-1 matrix of
> doubles, where n is the product of the dimensions. The values are stored
> column-by-column.
>
> So, if you extract one sub-column of this matrix, such as with A(i:j,p,q),
> this should be fast because data is stored locally, i.e. the next values in
> the extracted matrix is the neighbor of the previous values . If, on the
> other hand, you extract the 2nd or the 3d dimension (or more), for example,
> then I guess that this will be slower, because the hypermatrix has to gather
> values which are scattered in the internal n-by-1 matrix: there are many
> values in the internal matrix between two values in the extracted matrix.
>
> For those interested, here is the detail in Scilab v5. For a hypermatrix of
> doubles (or ints), the extraction operator is implemented in the compiled C
> source code: data_structures/src/c/hmops.c: intehm. This code is used in the
> statements :
>
> M = ones(3,3,3);
> B = M(1,2:3,2);
>
> For other types of values (e.g. strings), the extraction is implemented in
> overloading/macros/%hm_e.sci. This code is used in the statements:
>
> M = hypermat([2,2,2],["a","a","a","a","a","a","a","a"]);
> B = M(1,1:2,2);
>
> This also may lead to performance differences, in the sense that an
> hypermatrix of doubles is probably much faster than an hypermatrix of
> strings.
>
> In Scilab v6, the implementation is based on compiled source code for all
> types of contents. Anyway, data locality will limit the performances in v6
> the same way as it limits the performances in v5: when we extract values,
> scattered values have to be gathered the same way as before.
>
> A regular matrix of doubles (i.e. with 2 indices) will in general be much
> faster than an hypermatrix of doubles. All in all, Scilab was first designed
> for that...
>
> Best regards,
>
> Michaël Baudin
>
>
>
>
> Le 01/06/2011 16:17, Mike Page a écrit :
>
>> Hi,
>>
>>
>> I think you have encountered a general problem with hypermatrices.  They
>> are
>> quite fast to operate on when you work across some dimensions, but across
>> other dimensions they can be *VERY* slow.  I don't know why this is -
>> maybe
>> somebody who wrote the hypermatrix code could look at it.
>>
>> For instance, some time ago I had a need to extract a "slice" (that is a
>> matrix) from a hypermatrix.  This took a very long time, but when I
>> "transposed" the hypermatrix using the permute function, then it was much
>> faster.  I think if you arrange your dimensions so that the data to be
>> joined fits on the end of the existing storage then it will be faster.
>>
>> Regards,
>> Mike.
>>
>>
>> -----Original Message-----
>> From: Orbeman [mailto:ludo.wag at laposte.net]
>> Sent: 01 June 2011 14:29
>> To: users at lists.scilab.org
>> Subject: [scilab-Users] RE: Concatenation with hypermatrix
>>
>>
>> I've finally found the command C=cat(3,C,x). It works but very slowly. For
>> exemple it needs more than 2 min to create a 890 * 2 * 5 3D hypermatrix,
>> but
>> barely 1 sec to create the same data with 890 * (2+2+2+2+2) 2D matrix.
>>
>> In fact, I've analysed n experiences which data (2 rows and variable line)
>> are written in one file per experience. All of the files are not the same
>> length, so I add number 'tag' to egal the maximun number of line, here
>> 890.
>>
>> After that I've created some others (hyper) matrix with different size of
>> row : 890 * 1 * 5, 890 * 3 * 5,...
>>
>> I can imagine the increase of time for comptute.
>>
>> --
>> View this message in context:
>>
>> http://mailinglists.scilab.org/Concatenation-with-hypermatrix-tp3010503p3010
>> 714.html
>> Sent from the Scilab users - Mailing Lists Archives mailing list archive
>> at
>> Nabble.com.
>>
>> No virus found in this incoming message.
>> Checked by AVG - www.avg.com
>> Version: 9.0.901 / Virus Database: 271.1.1/3670 - Release Date: 05/31/11
>> 19:34:00
>>
>>
>
> --
> Michaël Baudin
> Ingénieur de développement
> michael.baudin at scilab.org
> -------------------------
> Consortium Scilab - Digiteo
> Domaine de Voluceau - Rocquencourt
> B.P. 105 - 78153 Le Chesnay Cedex
> Tel. : 01 39 63 56 87 - Fax : 01 39 63 55 94
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20110610/6812bd77/attachment.htm>