[Scilab-users] Optim & NelderMead use [closed]

[Pierre Vuillemin]

Hi Paul,

You should be careful when using the square root function as it is not
differentiable at 0 (cf the attached file which illustrates this point
in your case). This will leads to issues and prevent optim from "really"
(in a sound way) converging towards the solution. 

To address the issue, you can 

- reformulate your objective function so that it is smooth near the
solution (you will still have an issue at 0 where g is not
differentiable though), 

- use order 0 methods (still, some of them might fail on non-smooth
optimisation problems as the minimum might be very "narrow") 

- dig into non-smooth optimisation (see e.g.
http://napsu.karmitsa.fi/nso/)

Regards,

Pierre 

Le 16.01.2017 10:30, Carrico, Paul a écrit :

> Hi all 
> 
> After  performing tests (and modifying the target function as it should have been done first), I can better understand how to use 'optim' and 'Neldermead' procedures. 
> 
> For my needs the mean flags are :
> 
> -          Step h in numderivative à usefull reading as "EE 221 Numerical Computing" Scott Hudson
> 
> -          The threshold epsg in optim (%eps is the default value - such high accuracy is not necessary for my application - furthermore using a value such as 1e-5 leads to err=1 that is correct for checking)
> 
> -          Ditto for Nelder-Mead and '-tolfunrelative' & '-tolxrelative'
> 
> Now it works fine :-)
> 
> Thanks all for the support
> 
> Paul
> 
> #####################################################################
> 
> mode(0)
> 
> clear
> 
> global count_use;
> 
> count_use = 0;
> 
> _// ****_
> 
> function F=lineaire(X, A2, B2)
> 
> F = A2*X+B2;
> 
> endfunction
> 
> _// ****_
> 
> function G=racine(X, A1, B1)
> 
> G = sqrt(A1*X) + B1;
> 
> endfunction
> 
> _// ****_
> 
> function F=target(X, A1, B1, A2, B2)
> 
> val_lin = lineaire(X,A2,B2);
> 
> val_rac = racine(X,A1,B1);
> 
> F = sqrt((val_lin - val_rac)^2); 
> 
> global count_use;
> 
> count_use = count_use +1;
> 
> endfunction
> 
> _// Cost function: _
> 
> function [F, G, IND]=cost(X, IND, A1, B1, A2, B2)
> 
> F = target(X);   
> 
> _//    g = numderivative(target, x.',order = 4); _
> 
> G = numderivative(target, X.',1e-3, order = 4);  _// 1E-3 => see EE 221 "Numerical Computing" Scott Hudson    _
> 
> _// Study of the influence of h on the number of target function calculation & the fopt accuracy:_
> 
> _// (epsg = %eps here)_
> 
> _// h = 1.e-1 => number = 220 & fopt = 2.242026e-05_
> 
> _// h = 1.e-2 => number = 195 & fopt = 2.267564e-07_
> 
> _// h = 1.e-3 => number = 170 & fopt = 2.189495e-09 ++++++_
> 
> _// h = 1.e-4 => number = 190 & fopt = 1.941203e-11_
> 
> _// h = 1.e-5 => number = 215 & fopt = 2.131628e-13_
> 
> _// h = 1.e-6 => number = 235 & fopt = 0._
> 
> _// h = 1.e-7 => number = 255 & fopt = 7.105427e-15_
> 
> _// h = 1.e-8 => number = 275 & fopt = 0._
> 
> endfunction
> 
> _// *************************_
> 
> _// optimisation with optim_
> 
> initial_parameters = [10]
> 
> lower_bounds = [0];
> 
> upper_bounds = [1000];
> 
> nocf = 1000;      _// number max of call of f_
> 
> niter = 1000;    _// number max of iterations_
> 
> a1 = 30;
> 
> b1 = 2.5;
> 
> a2 = 1;
> 
> b2 = 2;
> 
> epsg = 1e-5;    _// gradient norm threshold (%eps by defaut) --> lead to err = 1 !!!_
> 
> _//epsg = %eps;     // lead to Err = 13_
> 
> epsf = 0;       _//threshold controlling decreasing of f (epsf = 0 by defaut)_
> 
> costf = list (cost, a1, b1, a2, b2);
> 
> [fopt, xopt, gopt, work, iters, evals, err] = optim(costf,'b',lower_bounds,upper_bounds,initial_parameters,'qn','ar',nocf,niter,epsg,epsf);
> 
> printf("Optimized value : %g\n",xopt);
> 
> printf("min cost function value (should be as closed as possible to 0) ; %e\n",fopt);
> 
> printf('Number of calculations = %d !!!\n',count_use);
> 
> _// Curves definition_
> 
> x = linspace(0,50,1000)';
> 
> plot_raci = racine(x,a1,b1);
> 
> plot_lin = lineaire(x,a2,b2);
> 
> scf(1);
> 
> drawlater();
> 
> xgrid(3);
> 
> f = gcf(); 
> 
> _//f                                          _
> 
> f.figure_size = [1000, 1000];                
> 
> f.background = color(255,255,255);           
> 
> a = gca();                                   
> 
> _//a                                          _
> 
> a.font_size = 2;                             
> 
> a.x_label.text = "X axis" ;                  
> 
> a.x_location="bottom";                       
> 
> a.x_label.font_angle=0;                      
> 
> a.x_label.font_size = 4;                     
> 
> a.y_label.text = "Y axis";
> 
> a.y_location="left";
> 
> a.y_label.font_angle=-90;
> 
> a.Y_label.font_size = 4;
> 
> a.title.text = "Title";                     
> 
> a.title.font_size = 5;
> 
> a.line_style = 1;    
> 
> _// Curves plot_
> 
> plot(x,plot_lin);
> 
> e1 = gce(); 
> 
> p1 = e1.children;                           
> 
> p1.thickness = 1;
> 
> p1.line_style = 1;
> 
> p1.foreground = 3;
> 
> plot(x,plot_raci);
> 
> e2 = gce(); 
> 
> p2 = e2.children;                           
> 
> p2.thickness = 1;
> 
> p2.line_style = 1;
> 
> p2.foreground = 2;
> 
> drawnow();
> 
> EXPORT CONTROL : 
> Cet email ne contient pas de données techniques
> This email does not contain technical data 
> 
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20170116/a72743e5/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nondiff.png
Type: image/png
Size: 6457 bytes
Desc: not available
URL: <https://lists.scilab.org/pipermail/users/attachments/20170116/a72743e5/attachment.png>