[Scilab-users] Performance issue with mget on inhomogeneous binary file

[Serge Steer]

Le 21/12/2012 11:46, phil_shvarcz a écrit :
> Hi all,
>
> being new to scilab I'm not sure if my problem has a trivial solution or
> not. I searched all documentation and forum postings but did not find
> anything relevant for my problem. So that's it:
>
> I need to read in quite big binary files (several 100MB) composed of
> successive pairs of float (4 byte word) and double (8 byte word). That is,
> the byte sequence looks like this ("ffff" is a 4-byte float and "dddddddd"
> is an 8 byte double):
> ffffddddddddffffddddddddffffddddddddffffdddddd
>
> if I use this code:
You forgot it?
>
>
>
> the file is read in at an incredibly slow rate (a few MB take minutes to be
> read in).
>
> If I read in the entire file in one flush
>
>
>
> it takes a second or so to read in 100MB. However, the data structure is
> lost and the vector vb contains nonsensical doubles.
>
> Is there a way in scilab to read in a (structureless) vector of bytes and
> cast it to the right structure afterwards, like one would do with an array
> in C ?
I do not think so.
>
> Besides, is it normal that mread is *so* slow when used to read a file in a
> word-by-word manner instead of reading it in one big chunk?
I do not know the mread function. Whre do you find it?

Any way it is normal that a word_by-word read being slower because on 
each iteration the program as to check if the arguments are ok or not, 
the system as to create an interrupt , ...

May be you can use a code  similar to the one below to be able to read 
your file. It is bit tricky because it needs to create an intermediate file



suppose your file is named yourfile and contains N floatting point numbers


//read all the numbers in the  file assumming float format
u=mopen('yourfile','r');
T=mget(3*N,'f',u);
F=T(1:3:$); // the float numbers
mclose(u);
//now write down all the records corresponding to doubles
I=(2:3)'.*.ones(1,N/2)+(0:3:3*N-3).*.ones(2,1);
u=mopen('/tmp/foo','w')
mput(T(I),'f',u);
mclose(u);
//read it again as double
u=mopen('/tmp/foo','r');
D=mget(N,'d',u);
mclose(u);


Serge Steer
>
> Thanks for reading,
> regards
> Phil
>
>
>
> --
> View this message in context: http://mailinglists.scilab.org/Performance-issue-with-mget-on-inhomogeneous-binary-file-tp4025574.html
> Sent from the Scilab users - Mailing Lists Archives mailing list archive at Nabble.com.
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20121221/7dbaaabc/attachment.htm>