[Scilab-users] parsing TSV (or CSV) file with scilab is a nightmare

[Antoine Monmayrant]

Hello David,


Thanks.
No I did not have a look at fscanfMat, as I had in mind to also import 
the header and footer.

Samuel and Jan also proposed to simply use "mgetl(fd)" to grab the whole 
file at once.
Their solution seems to work (or at least does not crash scilab on the 
first file I tested!).


Thank you all for your kind help,


Antoine



On 27/04/2020 18:06, CHEZE David 227480 wrote:
>
> Hi Antoine,
>
> did you also look at fscanfMat ? It's handy when  space or tab separators.
>
> regards,
>
> David
>
> ------------------------------------------------------------------------
> *De :* users [users-bounces at lists.scilab.org] de la part de Antoine 
> Monmayrant [antoine.monmayrant at laas.fr]
> *Envoyé :* lundi 27 avril 2020 17:40
> *À :* Users mailing list for Scilab
> *Objet :* [Scilab-users] parsing TSV (or CSV) file with scilab is a 
> nightmare
>
> Hi all,
>
>
> This is both a rant and desperate cry for help.
> I'm trying to parse some TSV data (tab separated data file) with 
> scilab and I cannot find a way to navigate around the minefield of 
> bugs present in meof/mgetl/mgetstr/csvRead.
>
> A bit of context: I need to load into scilab data generated by a 
> closed source software.
> The data is in the form of many TSV files (that I cannot share in 
> full, just some redacted bits) with a header and a footer.
> I don't want to hand modify these files or edit them in any way (I 
> need to keep this as portable as possible, so no sed/awk/grep...)
>
>
>     OPTION 1: csvRead
>
> That's the most intuitive solution, however, because of 
> http://bugzilla.scilab.org/show_bug.cgi?id=16391 and the presence of 
> more than 1 empty line in my header/footer, this crashes Scilab.
>
>
>     OPTION 2: hand parsing line by line using mgetl/meof
>
> I tried:
>
> filename="tsv.txt";
> [fd, err] = mopen(filename, 'rt');
> while ~meof(fd) do
>     txtline=mgetl(fd,1);
> end
> mclose(fd)
>
> Saddly, and contrary to what's written in "help mgetl", meof keeps on 
> returning 0, well passed the end of the file and the while never ends!
>
>
>     OPTION 3: hand parsing chunk by chunk using mgetstr/meof
>
> "help meof" does not confirm that meof should work with mgetl, but 
> mgetstr is specifically listed.
> I thus tried:
>
> filename="tsv.txt";
> [fd, err] = mopen(filename, 'rt');
> while ~meof(fd) do
>     txtchunk=mgetstr(80,fd);
> end
> mclose(fd)
>
> But thanks to http://bugzilla.scilab.org/show_bug.cgi?id=16419 this is 
> also crashing Scilab.
>
>
>     OPTION 4: Can anyone here help me with this?
>
> I am really running out of ideas.
> Did I miss some -hmm- obvious combination of available file parsing 
> scilab functions to achieve my goal?
> I have the feeling that it would have been faster for me to just learn 
> a totally new language that does not suck at parsing files than trying 
> to get it to work with scilab....
>
>
> Antoine
>
> (depressed)
>
>
>
> http://bugzilla.scilab.org/show_bug.cgi?id=16419
>
>
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20200428/e71e1d98/attachment.htm>