[Scilab-users] parsing TSV (or CSV) file with scilab is a nightmare

[Jan Åge Langeland]

Antoine

To find out how long the file is (although not strictly necessary) I 
normally use:

fid  =  mopen(datafile,'rb');
mseek(0,fid,'end');
lef=mtell(fid)
mseek(0,fid); Then you can read in the whole file byte by byte (or split 
it up if it is big) : data=mgeti(lef,'c',fid);

The rest is just looking for the different letters and sort based on that.

Jan


On 2020-04-27 17:40 PM, Antoine Monmayrant wrote:
>
> Hi all,
>
>
> This is both a rant and desperate cry for help.
> I'm trying to parse some TSV data (tab separated data file) with 
> scilab and I cannot find a way to navigate around the minefield of 
> bugs present in meof/mgetl/mgetstr/csvRead.
>
> A bit of context: I need to load into scilab data generated by a 
> closed source software.
> The data is in the form of many TSV files (that I cannot share in 
> full, just some redacted bits) with a header and a footer.
> I don't want to hand modify these files or edit them in any way (I 
> need to keep this as portable as possible, so no sed/awk/grep...)
>
>
>     OPTION 1: csvRead
>
> That's the most intuitive solution, however, because of 
> http://bugzilla.scilab.org/show_bug.cgi?id=16391 and the presence of 
> more than 1 empty line in my header/footer, this crashes Scilab.
>
>
>     OPTION 2: hand parsing line by line using mgetl/meof
>
> I tried:
>
> filename="tsv.txt";
> [fd, err] = mopen(filename, 'rt');
> while ~meof(fd) do
>     txtline=mgetl(fd,1);
> end
> mclose(fd)
>
> Saddly, and contrary to what's written in "help mgetl", meof keeps on 
> returning 0, well passed the end of the file and the while never ends!
>
>
>     OPTION 3: hand parsing chunk by chunk using mgetstr/meof
>
> "help meof" does not confirm that meof should work with mgetl, but 
> mgetstr is specifically listed.
> I thus tried:
>
> filename="tsv.txt";
> [fd, err] = mopen(filename, 'rt');
> while ~meof(fd) do
>     txtchunk=mgetstr(80,fd);
> end
> mclose(fd)
>
> But thanks to http://bugzilla.scilab.org/show_bug.cgi?id=16419 this is 
> also crashing Scilab.
>
>
>     OPTION 4: Can anyone here help me with this?
>
> I am really running out of ideas.
> Did I miss some -hmm- obvious combination of available file parsing 
> scilab functions to achieve my goal?
> I have the feeling that it would have been faster for me to just learn 
> a totally new language that does not suck at parsing files than trying 
> to get it to work with scilab....
>
>
> Antoine
>
> (depressed)
>
>
>
> http://bugzilla.scilab.org/show_bug.cgi?id=16419
>
>
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20200427/043e6e42/attachment.htm>