[Scilab-users] HDF5 save is super slow

[Clément DAVID]

Hello,

My 2cents, this is probably a poor man’s approach but Xcos offers vec2var / var2vec functions that encode in a double vector any Scilab datatypes passed as arguments. The encoding duplicates the data in memory so there might be some overhead.

On my machine, I have these timings using the attached script (Antoine’s one edited):
save list of syslins: 1.361704
save list of vec[]: 0.056788
save var2vec(list of syslins): 0.014411

Discarding hdf5 groups creation is a huge performance win but remove any way to create clean hdf5 (eg. to address subgroups directly).

Thanks,

--
Clément

From: users <users-bounces at lists.scilab.org> On Behalf Of Arvid Rosén
Sent: Tuesday, October 16, 2018 1:01 PM
To: antoine.monmayrant at laas.fr; Users mailing list for Scilab <users at lists.scilab.org>
Subject: Re: [Scilab-users] HDF5 save is super slow

From: users <users-bounces at lists.scilab.org<mailto:users-bounces at lists.scilab.org>> on behalf of "amonmayr at laas.fr<mailto:amonmayr at laas.fr>" <amonmayr at laas.fr<mailto:amonmayr at laas.fr>>
Reply-To: "antoine.monmayrant at laas.fr<mailto:antoine.monmayrant at laas.fr>" <antoine.monmayrant at laas.fr<mailto:antoine.monmayrant at laas.fr>>, Users mailing list for Scilab <users at lists.scilab.org<mailto:users at lists.scilab.org>>
Date: Tuesday, 16 October 2018 at 09:53
To: "users at lists.scilab.org<mailto:users at lists.scilab.org>" <users at lists.scilab.org<mailto:users at lists.scilab.org>>
Subject: Re: [Scilab-users] HDF5 save is super slow

Couldn't you create your own atom package that restore this raw memory dump for scilab 6.0?
I understand why we moved away from this model, but it seems to be key for you.
There is always a trade-off between portability (and robustness) and raw speed...

Yeah, if that was possible, I would certainly do it. We already have a bunch of C/C++ binaries that we compile and link dynamically, but for that to be easy to implement, I guess the lists and structures need to be stored linearly in one consecutive chunk of memory. I don’t know if that is the case. Anyone? C++ integrations and gateways are very poorly documented at the moment.
Otherwise, I would need to do some recursive implementation, that handles a bunch of different object types. Sounds painful.

Cheers,
Arvid
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20181018/2dff3721/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sample.sce
Type: application/octet-stream
Size: 761 bytes
Desc: sample.sce
URL: <https://lists.scilab.org/pipermail/users/attachments/20181018/2dff3721/attachment.obj>