[scilab-Users] Re: 500mo Ascii file

[Samuel Gougeon]

Hello,

Le 16/06/2012 17:05, Tingsten a écrit :
> Hello all,
>
> Following the "success" of reading a big file (around 200mo) I tried with
> bigger files ;-)
> Although I waited more than 5 minutes I was not able to load a 500mo file.
>
> So I wanted to only load 1 line every 100 but I did not find a good
> technique.
> I was thinking doing something like this (I noticed that a line as a
> "length" of 41 or 42) :
>
> f=mopen('my500mofile.txt','r');
> data=[];eof=0;
> While eof==0, data=[data;mgetl(f,1)];mseek(4100,'cur'); eof=meof(f);,end;
>
>
> mseek(4100,'cur'); should skip 100 line but the problem is that sometimes
> 4100 does not exactly correspond to exactly 100 lines and it messed up the
> result.....
>
> Any idee to avoid this?
>    

You may try this code:
-------------------
fid = file('open',"myFile.txt","old");
H = read(fid,5,1,"(a)");    // Header (5 lines)

// Sampling lines of data => Matrix
N = 100;    // Read 1 line over N
ok = %t;
L = [];
try
     L = read(fid,1,7,"(7f11.3)");
catch
     ok = %f;
end
while ok
     try
         read(fid,N-2,1,"(f11.3)");    // Skipped lines
         li = read(fid,1,7,"(7f11.3)")
         L = [L ; li ];
     catch
         ok = %f
     end
end
file('close',fid);
// Sampled lines are in L
-----------------
Please do not hesitate to report here the timing when applying it to 
your big file
(provided that all its data are supported by the f11.3 format. 
Otherwise, you may
adapt it in the script).

Regards

Samuel Gougeon