[Scilab-users] HDF5 save is super slow

[Stéphane Mottelet]

Hello Clément,

Le 18/10/2018 à 14:09, Clément DAVID a écrit :
>
> Hello,
>
> My 2cents, this is probably a poor man’s approach but Xcos offers 
> vec2var / var2vec functions that encode in a double vector any Scilab 
> datatypes passed as arguments. The encoding duplicates the data in 
> memory so there might be some overhead.
>
Do you think it would be complicated to continuously write the 
serialized data on the disk ?
>
> On my machine, I have these timings using the attached script 
> (Antoine’s one edited):
>
> save _list_ of _syslins_: 1.361704
>
> save _list_ of vec[]: 0.056788
>
> save var2vec(list of _syslins_): 0.014411
>
> Discarding hdf5 groups creation is a huge performance win but remove 
> _any way_ to create clean hdf5 (eg. to address subgroups directly).
>
> Thanks,
>
> --
>
> Clément
>
> *From:*users <users-bounces at lists.scilab.org> *On Behalf Of *Arvid Rosén
> *Sent:* Tuesday, October 16, _2018_ 1:01 PM
> *To:* antoine.monmayrant at laas.fr; Users mailing list for Scilab 
> <users at lists.scilab.org>
> *Subject:* Re: [Scilab-users] HDF5 save is super slow
>
> *From: *users <users-bounces at lists.scilab.org 
> <mailto:users-bounces at lists.scilab.org>> on behalf of 
> "amonmayr at laas.fr <mailto:amonmayr at laas.fr>" <amonmayr at laas.fr 
> <mailto:amonmayr at laas.fr>>
> *Reply-To: *"antoine.monmayrant at laas.fr 
> <mailto:antoine.monmayrant at laas.fr>" <antoine.monmayrant at laas.fr 
> <mailto:antoine.monmayrant at laas.fr>>, Users mailing list for Scilab 
> <users at lists.scilab.org <mailto:users at lists.scilab.org>>
> *Date: *Tuesday, 16 October 2018 at 09:53
> *To: *"users at lists.scilab.org <mailto:users at lists.scilab.org>" 
> <users at lists.scilab.org <mailto:users at lists.scilab.org>>
> *Subject: *Re: [Scilab-users] HDF5 save is super slow
>
> Couldn't you create your own atom package that _restore_ this raw 
> memory dump for _scilab_ 6.0?
> I understand why we moved away from this model, but it seems to be key 
> for you.
> There is always a trade-off between portability (and robustness) and 
> raw speed...
>
> Yeah, if that was possible, I would certainly do it. We already have a 
> bunch of C/C++ binaries that we compile and link dynamically, but for 
> that to be easy to implement, I guess the lists and structures need to 
> be stored linearly in one consecutive chunk of memory. I don’t know if 
> that is the case. Anyone? C++ integrations and gateways are very 
> poorly documented at the moment.
>
> Otherwise, I would need to do some recursive implementation, that 
> handles a bunch of different object types. Sounds painful.
>
> Cheers,
>
> Arvid
>
>
>
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> https://antispam.utc.fr/proxy/1/c3RlcGhhbmUubW90dGVsZXRAdXRjLmZy/lists.scilab.org/mailman/listinfo/users


-- 
Stéphane Mottelet
Ingénieur de recherche
EA 4297 Transformations Intégrées de la Matière Renouvelable
Département Génie des Procédés Industriels
Sorbonne Universités - Université de Technologie de Compiègne
CS 60319, 60203 Compiègne cedex
Tel : +33(0)344234688
http://www.utc.fr/~mottelet

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20181018/d40a3cd5/attachment.htm>