[Scilab-users] 'optim' vs 'Nelder-Mead' ... or difficulties to use 'optim'

[Stéphane Mottelet]

Hello 

Your target function is not differrentiable (Because of the absolute value). That explains why optim has some difficulties. Using a square instead should give the advantage to optim against nelder-mead.

S.



> Le 15 janv. 2017 à 11:39, paul.carrico at free.fr a écrit :
> 
> Hi all
> 
> Well, from a "macro" point of view, gradient based methods (order 1 method) should be more efficient than Nelder-Mead one (order 0) especially  for 1D topic, shouldn't be?
> 
> In the case here bellow, when I use 'Optim' it is not the case in comparison to 'NelderMead'... I don'y know what I'm doing wrong but I decide to frop off 'optim' in favor tp Nelder-Mead - nevertheless the later method requires much more iterations :-(
> 
> I'll be interested by some feedback on such topic, either in the mailing list (or in MP in order to not pollute it).
> 
> Paul
> ########################################"
> 'optim'
> ########################################
> mode(0)
> 
> function f=lineaire(x, a2, b2)
>     f = a2*x+b2;
> endfunction
> 
> function g=racine(x, a1, b1)
>     g = sqrt(a1*x) + b1;
> endfunction
> 
> function f=target(x)
>     val_lin = lineaire(x,1,2);
>     val_rac = racine(x,10,6);
>     f = abs(val_lin - val_rac);
> endfunction
> 
> // Cost function : 
> function [f, g, ind]=cost(x, ind)
>     f = target(x);   
> //    g = numderivative(target, x.'); 
> //    g = numderivative(target, x.',0.1,order = 2); 
>     g = numderivative(target, x.',order = 2);  // better but why ??? h automatic works better ???
> //    g = numderivative(target, x.',order = 4); 
> //    g = numderivative(target, x.',0.1, order = 4);
> endfunction
> 
> // optimisation avec optim
>     initial_parameters = [50]
>     lower_bounds = [0];
>     upper_bounds = [1000];
>     nocf = 1000;      // number of call of f
>     niter = 1000;    // number of iterations
>     [fopt, xopt, gopt, work, iters, evals, err] = optim(cost,'b',lower_bounds,upper_bounds,initial_parameters,'qn','ar',nocf,niter,imp=3);
>     xopt
>     fopt
>     err
> 
> // traçage courbes
> x = linspace(0,50,1000)';
> plot_raci = racine(x,10,6);
> plot_lin = lineaire(x,1,2);
> 
> scf(1);
> drawlater();
> xgrid(3);
> f = gcf(); 
> //f                                          
> f.figure_size = [1000, 1000];                
> f.background = color(255,255,255);           
> a = gca();                                   
> //a                                          
> a.font_size = 2;                             
> a.x_label.text = "X axis" ;                  
> a.x_location="bottom";                       
> a.x_label.font_angle=0;                      
> a.x_label.font_size = 4;                     
> a.y_label.text = "Y axis";
> a.y_location="left";
> a.y_label.font_angle=-90;
> a.Y_label.font_size = 4;
> a.title.text = "Title";                     
> a.title.font_size = 5;
> a.line_style = 1;    
> 
> // début des courbes
> plot(x,plot_lin);
> e1 = gce(); 
> p1 = e1.children;                           
> p1.thickness = 1;
> p1.line_style = 1;
> p1.foreground = 3;
> 
> plot(x,plot_raci);
> e2 = gce(); 
> p2 = e2.children;                           
> p2.thickness = 1;
> p2.line_style = 1;
> p2.foreground = 2;
> drawnow();
> 
> #######################################
> Nelder Mead
> #######################################
> mode(0)
> 
> function f=lineaire(x, a2, b2)
>     f = a2*x+b2;
> endfunction
> 
> function g=racine(x, a1, b1)
>     g = sqrt(a1*x) + b1;
> endfunction
> 
> function [f, index]=target(x, index)
>     val_lin = lineaire(x,1,2);
>     val_rac = racine(x,10,6);
>     f = abs(val_lin - val_rac);
> endfunction
> 
> // optimisation avec optim
>     initial_parameters = [50]
>     lower_bounds = [0];
>     upper_bounds = [1000];
> 
> nm = neldermead_new ();
> nm = neldermead_configure(nm,"-numberofvariables",1);
> nm = neldermead_configure(nm,"-function",target);
> nm = neldermead_configure(nm,"-x0",initial_parameters);
> nm = neldermead_configure(nm,"-maxiter",1000); 
> nm = neldermead_configure(nm,"-maxfunevals",1000); 
> nm = neldermead_configure(nm,"-tolfunrelative",10*%eps);
> nm = neldermead_configure(nm,"-tolxrelative",10*%eps);
> nm = neldermead_configure(nm,"-method","box");
> nm = neldermead_configure(nm,"-boundsmin",lower_bounds);
> nm = neldermead_configure(nm,"-boundsmax", upper_bounds);
> nm = neldermead_search(nm);
> nm = neldermead_restart(nm);
> xopt = neldermead_get(nm,"-xopt")
> fopt = neldermead_get(nm,"-fopt")
> nm = neldermead_destroy(nm);
> 
> // traçage courbes
> x = linspace(0,50,1000)';
> a1 = 10; b1 = 6;  // b1 > b2 here
> a2 = 1; b2 = 2;
> 
> plot_raci = racine(x,a1,b1);
> plot_lin = lineaire(x,a2,b2);
> 
> scf(1);
> drawlater();
> xgrid(3);
> f = gcf(); 
> //f                                          
> f.figure_size = [1000, 1000];                
> f.background = color(255,255,255);           
> a = gca();                                   
> //a                                          
> a.font_size = 2;                             
> a.x_label.text = "X axis" ;                  
> a.x_location="bottom";                       
> a.x_label.font_angle=0;                      
> a.x_label.font_size = 4;                     
> a.y_label.text = "Y axis";
> a.y_location="left";
> a.y_label.font_angle=-90;
> a.Y_label.font_size = 4;
> a.title.text = "Title";                     
> a.title.font_size = 5;
> a.line_style = 1;    
> 
> // début des courbes
> plot(x,plot_lin);
> e1 = gce(); 
> p1 = e1.children;                           
> p1.thickness = 1;
> p1.line_style = 1;
> p1.foreground = 3;
> 
> plot(x,plot_raci);
> e2 = gce(); 
> p2 = e2.children;                           
> p2.thickness = 1;
> p2.line_style = 1;
> p2.foreground = 2;
> drawnow();
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users