[Scilab-users] HDF5 save is super slow

[antoine monmayrant]

Le 18/10/2018 à 14:09, Clément DAVID a écrit :
> Hello,
>
> My 2cents, this is probably a poor man’s approach but Xcos offers vec2var / var2vec functions that encode in a double vector any Scilab datatypes passed as arguments. The encoding duplicates the data in memory so there might be some overhead.
Er, I tried var2vec, but it does not work with structures:
--> typeof(t)
  ans  =
  st

--> var2vec(t)
var2vec: Wrong type for input argument #1: Double, Integer, Boolean, 
String or List type.

Arghh... so var2vec does not work for any datatype right?

Antoine
>
> On my machine, I have these timings using the attached script (Antoine’s one edited):
> save list of syslins: 1.361704
> save list of vec[]: 0.056788
> save var2vec(list of syslins): 0.014411
>
> Discarding hdf5 groups creation is a huge performance win but remove any way to create clean hdf5 (eg. to address subgroups directly).
>
> Thanks,
>
> --
> Clément
>
> From: users <users-bounces at lists.scilab.org> On Behalf Of Arvid Rosén
> Sent: Tuesday, October 16, 2018 1:01 PM
> To: antoine.monmayrant at laas.fr; Users mailing list for Scilab <users at lists.scilab.org>
> Subject: Re: [Scilab-users] HDF5 save is super slow
>
> From: users <users-bounces at lists.scilab.org<mailto:users-bounces at lists.scilab.org>> on behalf of "amonmayr at laas.fr<mailto:amonmayr at laas.fr>" <amonmayr at laas.fr<mailto:amonmayr at laas.fr>>
> Reply-To: "antoine.monmayrant at laas.fr<mailto:antoine.monmayrant at laas.fr>" <antoine.monmayrant at laas.fr<mailto:antoine.monmayrant at laas.fr>>, Users mailing list for Scilab <users at lists.scilab.org<mailto:users at lists.scilab.org>>
> Date: Tuesday, 16 October 2018 at 09:53
> To: "users at lists.scilab.org<mailto:users at lists.scilab.org>" <users at lists.scilab.org<mailto:users at lists.scilab.org>>
> Subject: Re: [Scilab-users] HDF5 save is super slow
>
> Couldn't you create your own atom package that restore this raw memory dump for scilab 6.0?
> I understand why we moved away from this model, but it seems to be key for you.
> There is always a trade-off between portability (and robustness) and raw speed...
>
> Yeah, if that was possible, I would certainly do it. We already have a bunch of C/C++ binaries that we compile and link dynamically, but for that to be easy to implement, I guess the lists and structures need to be stored linearly in one consecutive chunk of memory. I don’t know if that is the case. Anyone? C++ integrations and gateways are very poorly documented at the moment.
> Otherwise, I would need to do some recursive implementation, that handles a bunch of different object types. Sounds painful.
>
> Cheers,
> Arvid