<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<font face="Arial">Hello,<br>
<br>
This message continues in a separate thread the discussion opened
@ <br>
<a class="moz-txt-link-freetext"
href="http://mailinglists.scilab.org/Release-Scilab-6-0-0-alpha-2-tp4033142p4033143.html">http://mailinglists.scilab.org/Release-Scilab-6-0-0-alpha-2-tp4033142p4033143.html</a><br>
<br>
It aims to test and assess interfaces of the new Scilab code <i>coverage</i>
module, that has been implemented without any Scilab Enhancement
Proposal.<br>
This module aims to replace </font><font face="Arial"><a
href="https://help.scilab.org/docs/5.5.2/en_US/section_f8ae48740240f0c0b0770316ec9938aa.html">profiling
functions</a></font><font face="Arial"> that are removed in
Scilab 6: <br>
</font>
<ul>
<li><font face="Arial">add_profiling() => covStart()<br>
</font></li>
<li><font face="Arial">reset_profiling() => no equivalent<br>
</font></li>
<li><font face="Arial">remove_profiling() => covStop()<br>
</font></li>
<li><font face="Arial">profile() => no
equivalent<br>
</font></li>
<li><font face="Arial">showprofile() => no equivalent<br>
</font></li>
<li><font face="Arial">plotprofile() => abandoned (actually,
without displaying the lines of code, it was hardly useful)</font><br>
</li>
</ul>
<p><font face="Arial">--------------------</font><br>
</p>
<font face="Arial"> </font>
<ul>
<li><font face="Arial"><b>"c</b><b>o</b><b>v</b><b>...</b><b>"
naming</b>: </font><font face="Arial">"cov" rather makes
thinking about <i>cov</i>ariance rather than coverage, </font><font
face="Arial">"code coverage" being an expression quite
specific to software engineering. It is not explicit for users
that would simply want to test and benchmark the code of their
"macros".<br>
</font><font face="Arial"><b>"codebench()</b>" is suggested as
an </font><font face="Arial">alternative name of a unique
function, with an action parameter: <b>codebench("start" ...)</b>,
codebench("write",..) etc.<br>
<br>
By the way, replacing former profiling functions needs as well
to replace their <b>timing feature</b> (cumulated time spent
to run each line of code), not only their coverage one (the
number of times that each line has been run). So, the same
profiling function should go beyond coverage, without
implementing extra specific functions.<br>
<b>Time counters</b> have been implemented : <a
class="moz-txt-link-freetext"
href="https://codereview.scilab.org/#/c/16397/"><a class="moz-txt-link-freetext" href="https://codereview.scilab.org/#/c/16397/">https://codereview.scilab.org/#/c/16397/</a></a><br>
</font><font face="Arial">Timing features look to be already
available in Scilab 6.0-b1, despite this commit hereabove has
not yet been included in the distrib. Strange.</font></li>
<br>
<li><font face="Arial"><b><a
href="https://help.scilab.org/docs/5.5.2/en_US/add_profiling.html">add_profiling()</a>
/ covStart(paths)</b>: <br>
</font></li>
<ul>
<li><font face="Helvetica, Arial, sans-serif"><b>covStart("isempty")
does</b><b> not work</b> (and does not yield any error
message), contrarily to what <i>help covStart</i> tells. It
should work. Currently, only <i>covStart(isempty)</i> is
able to yields results. Profiling functions shall be as well
console-oriented: input arguments of text type should be
accepted as well, like with former profiling functions.<br>
<br>
</font></li>
<li><font face="Arial"><b>covStart(libraryname)</b> allows to
set a whole library for profiling. This is an improvement.
With the former add_profiling(), it was only possible to set
functions one by one. But:</font><tt><br>
--> covStart(differential_equationlib)</tt><tt><br>
</tt><tt>covStart: Wrong type for input argument #1: A
two-columns string matrix expected.</tt><font
face="Helvetica, Arial, sans-serif"><br>
The documentation is not consistent with this requirement.
And this requirement is not consistent neither with the
function case (as isempty hereabove), nor with the following
(that is accepted):.<br>
</font><tt>--> covStart differential_equationlib </tt><tt><br>
</tt><tt>--></tt><font face="Helvetica, Arial, sans-serif"><br>
By the way, after that, no results are produced with
covWrite(..) (see below).<br>
<br>
</font></li>
<li><font face="Arial"><b>c</b><b>ovStart(paths)</b></font><font
face="Arial">: Since it is now possible to provide a file
path instead of a function name, we could think that it
becomes possible to get as well the <i>profile of scripts</i>
out of functions.</font><br>
<font face="Arial">However, from first tests, nothing is
produced by covWrite("html",".") when the covStart() has
been applied only to some scripts. The fact that scripts.sce
are excluded should be explicitly documented, unless
profiling is extended to them.<br>
<b><br>
</b></font></li>
</ul>
<li><font face="Arial"><b>covWri</b><b>t</b><b>e(..)</b>:</font></li>
<ul>
<li>
<div class="synopsis"><font face="Helvetica, Arial,
sans-serif"><a
href="https://help.scilab.org/docs/5.5.2/en_US/showprofile.html"><span
class="functionid">showprofile</span><span
class="default">(</span><span class="default">fun</span></a><span
class="default"><a
href="https://help.scilab.org/docs/5.5.2/en_US/showprofile.html">)</a>
has no replacement:<b>.</b></span></font></div>
<font face="Helvetica, Arial, sans-serif"> </font></li>
<ul>
<li><font face="Helvetica, Arial, sans-serif"><span
class="default"></span>Displaying profiling results in
the console is no longer possible. 95% of time, i used to
use this way to get and read results. It was the most
handy. It allowed to have the code edited in Scinotes and
related profiling scores displayed in the console aside.
Now, we have to switch between the web browser </font><font
face="Helvetica, Arial, sans-serif">and Scilab's IDE </font><font
face="Helvetica, Arial, sans-serif">(since results can now
be displayed only in HTML, after having found the right
file of results and clicked on it...) . Display in console
was straightforward and efficient, to improve the code
"on-the-fly", without to have to manage any HTML files.</font><br>
<font color="#993399" face="Arial"><b>imo, this limitation
to HTML is </b><b>a critical regression</b>.<br>
<br>
</font></li>
<li><font face="Arial">covWrite(..)<font color="#993399">
outputs results in HTML <i>after</i> each line of code,
instead of before each line number: <b>This is a
blocking regression</b>.</font><br>
It prevents any use of new profiling functions. This
trivial <a
href="http://bugzilla.scilab.org/show_bug.cgi?id=12807">bug
#12807</a> was as well for a long time somewhat killing
showprofile(). It is back. Why?<br>
Imagine that you manufacture suitcases, but the designer
has put the handle under them, between their wheels.
Currently, covWrite() is somewhat in this status. <br>
See attached uman.html.zip, and please tell me where is
the handle.<br>
<br>
</font></li>
<li><font face="Arial">outputs from covWrite() are
unclear/undocumented (see attached file, from the covWrite
basic example):</font></li>
<ul>
<li><font face="Arial">items in the header are not clearly
defined: </font>instrs, coverage rates, ..<br>
</li>
<li><font face="Arial">Both percentages for each line have
no legend</font></li>
<li><font face="Arial">Color codes (red/black) have no
legend</font></li>
<li><font face="Arial">red "x 0" on some lines have no legend</font></li>
<li><font face="Arial">Spent times are displayed with an irrelevant
relative accuracy (too many digits)<br>
<br>
</font></li>
</ul>
</ul>
<li><font face="Arial"><a
href="https://help.scilab.org/docs/5.5.2/en_US/profile.html">c
= profile(fun)</a> has no replacement:</font></li>
<ul>
<li><font face="Helvetica, Arial, sans-serif">It was possible
to get results in a matrix of text c. It is no longer the
case. <b>This is a medium regression</b>.</font><br>
<font face="Helvetica, Arial, sans-serif">
</font><br>
</li>
</ul>
<ul>
<li><font face="Arial">It was possible to get the profile of a
single chosen function (or of a subset of them with a for
loop) among all ones set for profiling. </font><br>
<font face="Arial">
It is no longer the case: We are now compelled either to
overwrite all previous results in the same given
destination directory, or to create a new destination
directory containing all results each time that we want
new results for a single function among all profiled ones.<b>
This is a major regression.<br>
<br>
</b></font></li>
</ul>
<li><font face="Helvetica, Arial, sans-serif"><b>covWrite() does</b><b>
not work on a library</b>: <br>
<font face="Courier New, Courier, monospace">covStop(); <br>
covStart("differential_equationlib"); // then run examples
from inttrap(), integrate() and intc() help pages<br>
covWrite("html", ".") // produces only a ./report.html
file that is empty.<br>
</font><br>
</font>
</li>
</ul>
<li><font face="Arial"><a
href="https://help.scilab.org/docs/5.5.2/en_US/reset_profiling.html">reset_profiling()</a>
has no replacement: It is no longer possible to reset counters
of a chosen function or of a set of chosen functions. <br>
To do so, stopping and restarting <b>all of them</b> is now
mandatory.<br>
<br>
</font></li>
<li><font face="Arial"><a
href="https://help.scilab.org/docs/5.5.2/en_US/remove_profiling.html">remove_profiling(fun)</a>
/ <b>covStop()</b>: </font></li>
<ul>
<li><font face="Arial">In one hand, remove_profiling(fun) did
not allow to stop profiling of all profiled functions. This
was unhandy. covStop() allows this. This is nice.</font></li>
<li><font face="Arial">In the other hand, covStop() does not
allow to stop profiling for a single function or a file or
an explicit set of files or a library. The only way to stop
profiling is now to stop it for <b>all</b> "started"
functions.<br>
It could be disturbing if profiling a function makes it
slower, or covWrite() cannot export resuls for a subset of
"started" functions. But does profiling a function make it
slower? Tests to be done.<br>
IMO, covWrite() and covStop(..) shall have the same input
modes as allowed with covStart().<br>
<br>
</font></li>
</ul>
<li><font face="Arial">Neither binary files of results nor
covMerge() that processes them have been tested. When simpler
usages will work...<br>
</font></li>
</ul>
<p><font face="Arial">-----------------</font><br>
</p>
<ul>
</ul>
<p><b><font face="Helvetica, Arial, sans-serif">Partial (and
personal) conclusions:</font></b></p>
<ul>
<li><font face="Helvetica, Arial, sans-serif">Removed profiling
functions need replacements.</font></li>
<li><font face="Helvetica, Arial, sans-serif">The new profiling
module has been designed without SEP. Just matching features
of former profiling functions would not have needed any SEP.
But this is not the situation at all. The new module is
clearly not ready. It is hardly an alpha release.<br>
</font></li>
</ul>
<p><font face="Helvetica, Arial, sans-serif">Other tests, comments
and users opinions are somewhat needed and welcome.</font></p>
<p><font face="Helvetica, Arial, sans-serif">Regards<br>
Samuel Gougeon<br>
</font><br>
</p>
<ul>
</ul>
</body>
</html>