[Scilab-Dev] Coverage ? <= Re: Release Scilab 6.0.0 alpha 2

[Clément David]

Hi Samuel,

Le dimanche 28 février 2016 à 20:24 +0100, Samuel Gougeon a écrit :
> Hello,
> 
> > * A new profiler and coverage tool has been added.
> >   It replaces the "profile" function (which is still there but not working, and will be removed
> > in the final release).
> >   It produces HTML report for the time spent (profile) on each line. Example of use:
> > 
> >      covStart(isempty) // instruments the "isempty" macro; instrumenting a whole library can be
> > done with: covStart("elementary_functions")
> >      for i=1:1e5; isempty(i); end // execute some code - this will be profiled
> >      covWrite("html", "TMPDIR/coverage") // write the profiling/coverage report as "report.html"
> > in the folder specified as second parameter
> >      covStop(); // stop the profiling
> > 
> >   A function to merge multiple profiler/coverage is provided too: covMerge([
> > list_of_files_to_merge], file_to_merge_to).
> >   The API will change before the release; the current API is mostly to get feedback from the
> > community.
> > 
> > We didn't want to advertise it too much, as we may changes these functions before the release...
> Here we are on the dev@ list. If even on the dev@ it is not possible to openly discuss about
> features in development, then where?

The dev@ ML is in fact the right place to discuss new features or how to improve things. On the
"coverage" feature especially it has been mainly developed by calixte as a tool to validate its
analysis pass (not as a well as a designed from scratch new feature).

On the team we asked ourself if we should integrate it or not as it was not in the roadmap for 6.0.0
but is useful enough to be present. We decided to provide it (as an incomplete dev) !

> This part of the release notes is very surprising, because it only and hardly starts what could be
> a proper Scilab Enhancement Proposal. 
> No SEP was published. This means, that, no, you don't really need our feedback, since SEP are
> mostly designed for that.
> For the moment, "coverage" does not look at all as a new functional feature.
> Among the 3 proposed syntaxes, the first one is (now?) bugged:
> covStart(isempty)    // yielded an error the first time i ran it. And when i tried
> covStart("isempty") // it did not complain. After that, i tried again, from another fresh session,
> or a running one, 
> covStart(isempty)    // and it did no longer complain. May be it will complain again later, who
> knows.
> 
> Mixing input types between strings (name of functions or here library) and functions pointers
> (covStart(isempty), but covStart("mylibrary")) were already misleading within the former set of
> profiling functions. This has been reported on bugzilla. The first announcement of the new set
> shows that we learn almost nothing about former design/usage complications.
>   
> Then i tried with covStart("strsubst") // that is a builtin. Are built-in profilable? I know that
> the former profiling applied "only" on Scilab macros, but since this is a new profiler, i do not
> assume anything about any possible restriction. I do not really expect any answer here. A SEP is
> the right place to define what does a function, with which kind of parameters, according to which
> parameters format (string or function or primitive), the naming, etc.  A SEP is also a very good
> start to build the related documentation.
> 
> Whatever "strsubst" is an acceptable entry or not, no error is produced.
> 
> After running "uman" that uses a lot of strsubst(), covWrite(..) produces an empty report
> (attached).
> Same thing with covStart(strsubst)
> Same thing with covStart(uman) or covStart("uman") // that is a macro
> Nothing. I get nothing with the beta-1. Unfortunately, i have uninstalled the alpha-2 for which
> these "coverage" functions have been first announced. The announcement has not changed for the
> beta-1.
> 
> The former profiling functions had an output in console, as well as in argout to enable post-
> processing (as for the running upgrade of bench_run(), >10 (?) years after its first release) , or
> in an interactive plot.
> The few that is announced for the new profiling features looks much less than what was formerly
> available.
>  
> So, as somewhat Antoine told it, and after your answer, what i understand is that S/E needs to
> tell that it needs our feedback. I guess that we are expected to believe it. For the moment
> nothing really tell us that it needs more than telling it. But this can change.
> 
> Thanks, and regards.
> 
> Samuel Gougeon
> One of your 100000 industrial downloaders/month

Thanks for this reports, I added some documentation [1] and will try to have a scilab backend (to
provide information back to scilab is a similar way as before).

Do not hesitate to post bugs about arguments management, there might as (as written before) this was
just an internal tool initially.

[1]: http://cgit.scilab.org/scilab/commit/?id=5af4b83932053c80634254cdd3b2d1a312d5e338

Regards,

--
Clément