<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Rafael,<br>
<br>
you are right: when the magnitude of the outliers is too large,
lsq_splin is not always good. But if you increase the number of
points, you could have something better (with the same example (ie
50 rather than 5) replace x by x = %pi * [-1:0.01:1]').<br>
<br>
Best regards<br>
<br>
Calixte<br>
<br>
On 05/03/2013 04:15, Rafael Guerra wrote:
<blockquote cite="mid:DUB101-ds8B6AA192F72375C6BE65BCCFB0@phx.gbl"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.5pt;
font-family:Consolas;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:Consolas;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Hi
Calixte,<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Thanks
for your feedback, the beatiful example and very useful
script.<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">The
weighted least squares cubic spline fitting function
(lsq_splin) seems to work very well for outliers of moderate
magnitude. However, it seems to fail for larger outliers:<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:10.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:10.0pt;font-family:"Courier New"">//
Replace for instance the three outliers defined in your
script by:<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:10.0pt;font-family:"Courier New"">z(10,10)=50;<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:10.0pt;font-family:"Courier New"">z(30,20)=50;<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:10.0pt;font-family:"Courier New"">z(10,40)=50;<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Sinha
and Schunck (1992) address this problem using a two-stage
procedure, where a robust local algorithm MLMS (moving least
median of squares of error) is run first to get rid of the
outliers. Thereafter, weighted least squares cubic splines
are used. MLMS seems very computationally intensive.<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">
<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">A
different approach is the Loess method - see for instance
W.G. Jacoby (2000).<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Best
regards,<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Rafael
G.<o:p></o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoPlainText"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p> </o:p></span></p>
<p class="MsoPlainText">-----Original Message-----<br>
From: <a class="moz-txt-link-abbreviated" href="mailto:users-bounces@lists.scilab.org">users-bounces@lists.scilab.org</a>
[<a class="moz-txt-link-freetext" href="mailto:users-bounces@lists.scilab.org">mailto:users-bounces@lists.scilab.org</a>] On Behalf Of Calixte
Denizet<br>
Sent: Monday, March 04, 2013 4:01 PM<br>
To: <a class="moz-txt-link-abbreviated" href="mailto:users@lists.scilab.org">users@lists.scilab.org</a><br>
Subject: Re: [Scilab-users] Surface smoothing in Scilab,
immune to outliers<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Hi Rafael,<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">You can try to use lsq_splin along the
axes x and y.<o:p></o:p></p>
<p class="MsoPlainText">It seems to work on the following
example:<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">x = %pi * [-1:0.05:1]';<o:p></o:p></p>
<p class="MsoPlainText">z = sin(x)*cos(x)';<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">f = gcf();<o:p></o:p></p>
<p class="MsoPlainText">f.color_map = jetcolormap(32);<o:p></o:p></p>
<p class="MsoPlainText">subplot(131);<o:p></o:p></p>
<p class="MsoPlainText">xtitle("Exact values");<o:p></o:p></p>
<p class="MsoPlainText">plot3d(x, x, z, 70, 70);<o:p></o:p></p>
<p class="MsoPlainText">e=gce();<o:p></o:p></p>
<p class="MsoPlainText">e.color_flag = 1;<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">z = z + 0.5*rand(z);<o:p></o:p></p>
<p class="MsoPlainText">z(10,10)=5;<o:p></o:p></p>
<p class="MsoPlainText">z(30,20)=5;<o:p></o:p></p>
<p class="MsoPlainText">z(10,40)=5;<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">subplot(132);<o:p></o:p></p>
<p class="MsoPlainText">xtitle("Perturbated values");<o:p></o:p></p>
<p class="MsoPlainText">plot3d(x, x, z, 70, 70);<o:p></o:p></p>
<p class="MsoPlainText">e=gce();<o:p></o:p></p>
<p class="MsoPlainText">e.color_flag = 1;<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">zz=zeros(z);<o:p></o:p></p>
<p class="MsoPlainText">w=linspace(-%pi,%pi,5)';<o:p></o:p></p>
<p class="MsoPlainText">for i=1:size(z,'r')<o:p></o:p></p>
<p class="MsoPlainText">[y,d]=lsq_splin(x', z(i,:), w);<o:p></o:p></p>
<p class="MsoPlainText">zz(i,:)=interp(x', w, y, d);<o:p></o:p></p>
<p class="MsoPlainText">end<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">for i=1:size(zz,'c')<o:p></o:p></p>
<p class="MsoPlainText">[y,d]=lsq_splin(x, zz(:,i), w);<o:p></o:p></p>
<p class="MsoPlainText">zz(:,i)=interp(x, w, y, d);<o:p></o:p></p>
<p class="MsoPlainText">end<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">subplot(133);<o:p></o:p></p>
<p class="MsoPlainText">xtitle("Smoothed values");<o:p></o:p></p>
<p class="MsoPlainText">plot3d(x, x, zz, 70, 70);<o:p></o:p></p>
<p class="MsoPlainText">e=gce();<o:p></o:p></p>
<p class="MsoPlainText">e.color_flag = 1;<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Best regards<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Calixte<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">On 04/03/2013 04:36, Rafael Guerra
wrote:<o:p></o:p></p>
<p class="MsoPlainText">> Hello,<o:p></o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText">> Does somebody know if there are
Scilab functions capable of replacing <o:p></o:p></p>
<p class="MsoPlainText">> outliers via some local robust
fitting in 2D, i.e., that smooths <o:p></o:p></p>
<p class="MsoPlainText">> experimental data z=f(x,y) and is
immune to strong outliers.<o:p></o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText">> PS: CASCI in Atoms has a lowess
function which does this via local <o:p></o:p></p>
<p class="MsoPlainText">> robust linear fitting but for
functions of one variable only.<o:p></o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText">> Thanks and regards,<o:p></o:p></p>
<p class="MsoPlainText">> Rafael Guerra<o:p></o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText">><o:p> </o:p></p>
<p class="MsoPlainText">> --<o:p></o:p></p>
<p class="MsoPlainText">> View this message in context: <o:p></o:p></p>
<p class="MsoPlainText">> <a moz-do-not-send="true"
href="http://mailinglists.scilab.org/Surface-smoothing-in-Scilab-immune-to-o"><span
style="color:windowtext;text-decoration:none">http://mailinglists.scilab.org/Surface-smoothing-in-Scilab-immune-to-o</span></a><o:p></o:p></p>
<p class="MsoPlainText">> utliers-tp4026119.html Sent from
the Scilab users - Mailing Lists <o:p></o:p></p>
<p class="MsoPlainText">> Archives mailing list archive at
Nabble.com.<o:p></o:p></p>
<p class="MsoPlainText">>
_______________________________________________<o:p></o:p></p>
<p class="MsoPlainText">> users mailing list<o:p></o:p></p>
<p class="MsoPlainText">> <a moz-do-not-send="true"
href="mailto:users@lists.scilab.org"><span
style="color:windowtext;text-decoration:none">users@lists.scilab.org</span></a><o:p></o:p></p>
<p class="MsoPlainText">> <a moz-do-not-send="true"
href="http://lists.scilab.org/mailman/listinfo/users"><span
style="color:windowtext;text-decoration:none">http://lists.scilab.org/mailman/listinfo/users</span></a><o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">--<o:p></o:p></p>
<p class="MsoPlainText">Calixte Denizet<o:p></o:p></p>
<p class="MsoPlainText">Software Development Engineer<o:p></o:p></p>
<p class="MsoPlainText">-----------------------------------------------------------<o:p></o:p></p>
<p class="MsoPlainText">Scilab Enterprises<o:p></o:p></p>
<p class="MsoPlainText">143bis rue Yves Le Coz - 78000
Versailles, France <a moz-do-not-send="true"
href="http://www.scilab-enterprises.com"><span
style="color:windowtext;text-decoration:none">http://www.scilab-enterprises.com</span></a><o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">_______________________________________________<o:p></o:p></p>
<p class="MsoPlainText">users mailing list<o:p></o:p></p>
<p class="MsoPlainText"><a moz-do-not-send="true"
href="mailto:users@lists.scilab.org"><span
style="color:windowtext;text-decoration:none">users@lists.scilab.org</span></a><o:p></o:p></p>
<p class="MsoPlainText"><a moz-do-not-send="true"
href="http://lists.scilab.org/mailman/listinfo/users"><span
style="color:windowtext;text-decoration:none">http://lists.scilab.org/mailman/listinfo/users</span></a><o:p></o:p></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:users@lists.scilab.org">users@lists.scilab.org</a>
<a class="moz-txt-link-freetext" href="http://lists.scilab.org/mailman/listinfo/users">http://lists.scilab.org/mailman/listinfo/users</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Calixte Denizet
Software Development Engineer
-----------------------------------------------------------
Scilab Enterprises
143bis rue Yves Le Coz - 78000 Versailles, France
<a class="moz-txt-link-freetext" href="http://www.scilab-enterprises.com">http://www.scilab-enterprises.com</a></pre>
</body>
</html>