Better stick with DFT, smoothed DFT or try seasonal adjustment freeware Demetra+ - that's what official statisticians might do.<br><br>gin<br><br><br><div class="gmail_quote">On Mon, Nov 21, 2011 at 10:00 AM, Schreckenbach Stephan <span dir="ltr"><<a href="mailto:s.schreckenbach@truma.com">s.schreckenbach@truma.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<u></u>
<div link="blue" vlink="blue" lang="DE">
<div>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">Filtering temporal spikes is a good idea, since there are some of them. I will try that.<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">The data sample as around 7000 data points, the frequency I look for is around 1/10 * sample rate.<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB"><u></u> <u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">May be there are methods that are better suited for identifying frequency components in that kind of data?<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">FFT always describes the time series by harmonic oszillations, which might not work well<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">if oscillations are not (strictly) harmonic.<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB"><u></u> <u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">What about wavelets (don’t know much about it yet, though)?<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB"><u></u> <u></u></span></font></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB"><u></u> <u></u></span></font></p>
<div>
<div>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">Stephan</span></font><font color="navy"><span style="color:navy" lang="EN-GB"><u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman" color="navy"><span style="font-size:12.0pt;color:navy" lang="EN-GB"> <u></u><u></u></span></font></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman" color="navy"><span style="font-size:12.0pt;color:navy" lang="EN-GB"> </span></font><span lang="EN-GB"><u></u><u></u></span></p>
</div>
</div>
<div>
<div class="MsoNormal" style="text-align:center" align="center"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">
<hr size="2" width="100%" align="center">
</span></font></div>
<p class="MsoNormal"><b><font size="2" face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma;font-weight:bold">Von:</span></font></b><font size="2" face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma"> Charles Warner [mailto:<a href="mailto:cwarner.cw711@gmail.com" target="_blank">cwarner.cw711@gmail.com</a>]
<br>
<b><span style="font-weight:bold">Gesendet:</span></b> Samstag, 19. November 2011 05:12</span></font></p><div><div><font size="2" face="Tahoma"></font></div><div class="h5"><font size="2" face="Tahoma"><br>
<b><span style="font-weight:bold">An:</span></b> <u></u><a href="mailto:users@lists.scilab.org" target="_blank">users@lists.scilab.org</a><u></u><br>
<b><span style="font-weight:bold">Betreff:</span></b> Re: [scilab-Users] saisonality in time series</font></div></div><u></u><u></u><p></p>
</div><div><div></div><div class="h5">
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Another trick I have found that greatly reduces FFT noise it to temporarily mask any localized "spikes" in the data (such spikes, with a narrow
temporal profile have a very broad spectral distribution). One can also try to eliminate any offset by subtracting the mean (or the geometric mean or harmonic mean- the appropriate mean would be dictated by the nature of the data). This should hopefully
reduce the scale of the FFT amplitude, making it easier to spot any (especially low-frequency, or seasonal) potential frequency components.<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Fri, Nov 18, 2011 at 3:09 AM,
<u></u>Schreckenbach Stephan<u></u> <<a href="mailto:s.schreckenbach@truma.com" target="_blank">s.schreckenbach@truma.com</a>> wrote:<u></u><u></u></span></font></p>
<div link="blue" vlink="blue">
<div>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy">Hi,</span></font><u></u><u></u></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy"> </span></font><u></u><u></u></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">sorry, of course I meant seasonality.</span></font><u></u><u></u></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">The time series consists of longer term trends, short term noise
and short time seasonality. </span></font><u></u><u></u></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">oscillations / seasonality, if any, it is most likely to be nonharmonic.
I look for distinct frequencies.</span></font><u></u><u></u></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">When I did a FFT plot of the original time series there was noise
only in the spectrum.</span></font><u></u><u></u></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">I will give it a run with the differenciated series / the log
of the data. </span></font><u></u><u></u></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB">There is still the question how to test for significance of the
found seasonality. </span></font><u></u><u></u></p>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy" lang="EN-GB"> </span></font><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><font size="2" face="Arial" color="navy"><span style="font-size:10.0pt;font-family:Arial;color:navy">Stephan</span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman" color="navy"><span style="font-size:12.0pt;color:navy"> </span></font><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman" color="navy"><span style="font-size:12.0pt;color:navy"> </span></font><u></u><u></u></p>
</div>
</div>
<div>
<div class="MsoNormal" style="text-align:center" align="center"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">
<hr size="2" width="100%" align="center">
</span></font></div>
<p class="MsoNormal"><b><font size="2" face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma;font-weight:bold">Von:</span></font></b><font size="2" face="Tahoma"><span style="font-size:10.0pt;font-family:Tahoma">
Charles Warner [mailto:<a href="mailto:cwarner.cw711@gmail.com" target="_blank">cwarner.cw711@gmail.com</a>]
<br>
<b><span style="font-weight:bold">Gesendet:</span></b> Freitag, 18. November 2011 00:34<br>
<b><span style="font-weight:bold">An:</span></b> <a href="mailto:users@lists.scilab.org" target="_blank">
users@lists.scilab.org</a><br>
<b><span style="font-weight:bold">Betreff:</span></b> Re: [scilab-Users] saisonality in time series</span></font><u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Although "seasonality" is not the term I use for long term trends hidden in noisy data, I have had some success by
taking the log of the data, and running an FFT on the log data. Usually, I have some prior knowledge of the long-term periodic trends I expect, so it is relatively easy to determine quickly if this method works. Plotting the log of the data also gives one
a good feel for whether the data is stationary, or whether there are windows of data that can be treated as stationary. Any changing magnitude effect is, of course, reduced when on works with logs, but such effects can help one understand what the raw data
is really telling you.<br>
<br>
Charlie<u></u><u></u></span></font></p>
<div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">On Thu, Nov 17, 2011 at 12:40 PM, Mike Page <<a href="mailto:Mike@page-one.waitrose.com" target="_blank">Mike@page-one.waitrose.com</a>>
wrote:<u></u><u></u></span></font></p>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt">Hi,<br>
<br>
I don't know much about this application, but the Cepstrum can be used to<br>
find hidden periodicity in time series. Might be worth trying? I have used<br>
it for finding rotational components in the vibration signatures from<br>
rotating machinery. There's a simple example here<br>
(<a href="http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf" target="_blank">http://www.dliengineering.com/downloads/cepstrum%20analysis.pdf</a>).<br>
<font color="#888888"><span style="color:#888888"><br>
Mike.</span></font><u></u><u></u></span></font></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><br>
<br>
-----Original Message-----<br>
From: Petter Wingren [mailto:<a href="mailto:petterwr@gmail.com" target="_blank">petterwr@gmail.com</a>]<br>
Sent: 17 November 2011 17:18<br>
To: <a href="mailto:users@lists.scilab.org" target="_blank">users@lists.scilab.org</a><br>
Subject: Re: [scilab-Users] saisonality in time series<br>
<br>
<br>
Did a quick search but couldnt find anything obvious. I suppose the<br>
word you are looking for is seasonality - maybe that helps in finding<br>
something useful.<br>
<br>
On Thu, Nov 17, 2011 at 3:36 PM, <u></u>Schreckenbach Stephan<u></u><br>
<<a href="mailto:s.schreckenbach@truma.com" target="_blank">s.schreckenbach@truma.com</a>> wrote:<br>
><br>
> Hi,<br>
><br>
> I look for a test of saisonality in time series.<br>
> The time series might be instationary and nonlinear and the saisonality<br>
> / oscillation might have a changing amplitude. Furthermore the<br>
> distribution<br>
> might be unknown as well.<br>
> I need something to test for significant saisonality without knowing /<br>
> estimating a (linear) model of the time series.<br>
><br>
> ideas I got so far: Chi Square Test for independency:<br>
> I could test for independence of saison and mean value of the data<br>
><br>
> Chi Square Test to test for different means of two data groups.<br>
> I could test for a difference of the mean between several seasons.<br>
><br>
> Any more or better ideas?<br>
><br>
> Thanks in advance, Stephan<br>
><br>
><u></u><u></u></span></font></p>
</div>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"> <u></u><u></u></span></font></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><font size="3" face="Times New Roman"><span style="font-size:12.0pt"><u></u> <u></u></span></font></p>
</div></div></div>
</div>
</blockquote></div><br>