<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Reposted with pdf figure instead of too big scg<br>
<br>
JPD<br>
<br>
<div class="moz-forward-container"><br>
<br>
-------- Message original --------
<table class="moz-email-headers-table" border="0" cellpadding="0"
cellspacing="0">
<tbody>
<tr>
<th valign="BASELINE" align="RIGHT" nowrap="nowrap">Sujet: </th>
<td>Optimization</td>
</tr>
<tr>
<th valign="BASELINE" align="RIGHT" nowrap="nowrap">Date : </th>
<td>Thu, 15 Nov 2012 18:27:47 -0500</td>
</tr>
<tr>
<th valign="BASELINE" align="RIGHT" nowrap="nowrap">De : </th>
<td>Jean-Pierre Dussault
<a class="moz-txt-link-rfc2396E" href="mailto:Jean-Pierre.Dussault@Usherbrooke.CA"><Jean-Pierre.Dussault@Usherbrooke.CA></a></td>
</tr>
<tr>
<th valign="BASELINE" align="RIGHT" nowrap="nowrap">Pour : </th>
<td><a class="moz-txt-link-abbreviated" href="mailto:dev@lists.scilab.org">dev@lists.scilab.org</a></td>
</tr>
</tbody>
</table>
<br>
<br>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
Hi all,<br>
<br>
I am preparing examples for an optimization course for students in
image science. I use an example from <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://www.ceremade.dauphine.fr/%7Epeyre/numerical-tour/tours/optim_1_gradient_descent/">http://www.ceremade.dauphine.fr/~peyre/numerical-tour/tours/optim_1_gradient_descent/</a>
to promote the use of better algorithms than the simple gradient
descent. <br>
<br>
I attach the convergence plot of the norm of the gradient for 5
variants of the optim command: gc unconstrained, gc with bounds
[-%inf,%inf], gc with bounds [0,1], gc with bounds [0,%inf] and
nd. I also include the gradient descent.<br>
<br>
Except for the [0,%inf] variant, the solution has all components
strictly in [0,1] as displayed here:<br>
<blockquote>-->[max(xoptS),max(xoptGC),max(xoptGCB),max(xoptGCBinf),max(xoptGCB0inf),max(xoptND)]<br>
ans =<br>
<br>
0.9249840 0.9211455 0.9216067 0.9213056
1.0402906 0.9212348 <br>
<br>
-->[min(xoptS),min(xoptGC),min(xoptGCB),min(xoptGCBinf),min(xoptGCB0inf),min(xoptND)]<br>
ans =<br>
<br>
0.0671743 0.0718204 0.0678885 0.0714951
0.0772300 0.0714255 <br>
<br>
</blockquote>
On the convergence plot, we clearly see that the gradient norm of
the gc with [0,1] bounds stalls away from zero while with no
bounds or infinite bounds, it converges to zero. This is even more
severe for the variant with bounds [0.%inf], which no more
approaches the solution, making virtually no progress at all after
some 30 function evaluations.<br>
<br>
Is it a Scilab bug or a bad example for the gcbd underlying
routine? The cost function is strongly convex of dimension 65536.
Has someone experienced a similar behavior? <br>
<br>
<br>
This is unfortunate since I wish to convince my students to use
suitably constrained models instead of enforcing constraints
afterward. <br>
<br>
Thanks for any suggestion to work around this troublesome
situation.<br>
<br>
JPD<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
</div>
<br>
</body>
</html>