[Scilab-users] How to speed up making big structure?

[Samuel Gougeon]

Hello,

Le 03/07/2015 09:42, jaipur a écrit :
> I'm making a big structure (about 110,000 items) by reading data from
> Hipparcos star catalog text file.
> My procedure is as follows.
>
> MyStruct = struct(); Index = 0;
> while ~meof(Fd) do
>       ........
>       ........
>       Index = Index + 1;
>       MyStruct(Index).Field1 = ...;
>       MyStruct(Index).Field2 = ...;
>       MyStruct(Index).Field3 = ...;
> end
>
> But this procedure takes huge time!!
> The number of items is known. Could you teach me speedy way to make big
> structure?
> For example, keep memory for structure before starting like
>
> MyVector = ones(110000,1);
When creating new components of a structures array elements with an 
*increasing* index, the time spent to create each one increases, and the 
total time increases in a parabolic way:

-->clear s, tic; t=[]; for i=1:1000, s(i).n=%pi; if pmodulo(i,10)==0, 
t(i/10)=toc(); end, end, toc()
  ans  =
     52.603
-->plot(t)


Now, if components are created with a decreasing index, the time spent 
to create each one is constant, and the total time varies in a linear way:
-->clear s, tic; t=[]; for i=1000:-1:1, s(i).n=%pi; if pmodulo(i,10)==0, 
t(i/10)=toc(); end, end, toc()
  ans  =
     5.553
-->plot(t)


Finally, initializing the array to its final size, and then filling its 
components at increasing index is also fast and linear in time:

-->clear s, tic; t=[]; s(1000).n=0; for i=1:1000, s(i).n=%pi; if 
pmodulo(i,10)==0, t(i/10)=toc(); end, end, toc()
  ans  =
     5.696


Therefore, in your case, you may initialize:

      MyStruct(110000).Field1 = 0;
      MyStruct(110000).Field2 = 0;
      MyStruct(110000).Field3 = 0;

and then assign the components as you did, at increasing index.

HTH
Samuel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20150703/0b88e94f/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: diiibeji.png
Type: image/png
Size: 3954 bytes
Desc: not available
URL: <https://lists.scilab.org/pipermail/users/attachments/20150703/0b88e94f/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: beefafbe.png
Type: image/png
Size: 3467 bytes
Desc: not available
URL: <https://lists.scilab.org/pipermail/users/attachments/20150703/0b88e94f/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bedfjjhj.png
Type: image/png
Size: 3376 bytes
Desc: not available
URL: <https://lists.scilab.org/pipermail/users/attachments/20150703/0b88e94f/attachment-0002.png>