[scilab-Users] Re: RE [scilab-Users] Re: 500mo Ascii file

[CRETE Denis]

Hello,
As David mentioned, you can use regexp in SED (I am not familiar with perl-style regexp used in SciLab)
If you have the basic "sed" installed (standard with Unix/Linux OSes) and available also for Windows, it may be very fast to use  
Extracted_lines=unix_g('sed  -ne ""'+filter+'p;"" my500mofile.txt');

Where filter is the string for the selection of the lines to extract, e.g. starting at line 6, keep only the first 2 fields separated by comma:
6,$ s/^\([^,]*,[^,]*\).*$/\1/          

Giving
Extracted_lines=unix_g('sed  -ne ""6,$ s/^\([^,]*,[^,]*\).*$/\1/p;"" my500mofile.txt');

Then you can use
data=[];
for line=Extracted_lines', data=[data;eval(tokens(line,",")')];end;

or 
[Iter,Time,CHX]= msscanf(-1,Extracted_lines,'%f,%f');

HTH
Denis

-----Message d'origine-----
De : Tingsten [mailto:remi.mercier at imelavi.fr] 
Envoyé : mardi 19 juin 2012 14:33
À : users at lists.scilab.org
Objet : [scilab-Users] Re: RE [scilab-Users] Re: 500mo Ascii file

Hello Samuel and Michael,

Thx you a lot for your help.

First let's say that I found a reasonnable technique to do the job which is
:

f=mopen('my500mofile.txt','r');
data=mgetl(f,1);eof=0;
While eof==0,* mgetl(f,1);* data=[data;mgetl(f,1)];mseek(4100,'cur');
eof=meof(f);,end; 

By adding a mgetl(f,1) that I "throw away" I have no more the issue of recording a not complete line. So it does work with my 500mo file. Then by changing the 4100 we can adapt to different type (and size) of file.

Regarding the script you send, I did try Samuel's that look great. For the moment it goes out of the loop and goes false but I'll investigate why and try it. It could be much more efficient than mine (only things is that you read every line). 

Michael, I'll also try your technique.

Have a good day,
Rémi

--
View this message in context: http://mailinglists.scilab.org/500mo-Ascii-file-tp4024324p4024389.html
Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com.

--
To unsubscribe from this mailing-list, please send an empty mail to users-unsubscribe at lists.scilab.org
To check the archives of this mailing list, see http://mailinglists.scilab.org/