[Scilab-users] Using GROCER ms_var parameters for forecasting

[Brian Bouterse]

Hi Eric,

Thanks for the reply! Yes you understand my goals correctly, but one
clarification: It would be better to have the estimated values directly
instead of the filtered state probabilities. I usually get these with
ms_forecast(r, n).

I've been reading through the grocer code to determine how to write the
function you suggest. I do need it sooner than a few weeks so I'm
attempting to do it. It seems straightforward except for the y_hat, x_hat,
and z_hat variables I need to provide to MSVAR_Filt.(). Here are some
questions:

1) You say I need to feed MSVAR_Filt() with y_hat, x_hat, and z_hat, but
the variables in the function signature for MSVAR_Filt read
as y_mat,x_mat,z_mat. Did you mean y_mat or y_hat?

2) y_hat (2nd output) is an output of MSVAR_Filt(). The function comments
say that is my estimated y. Is that the direct estimates that I am looking
for?

3) I read through ms_var() to see how to derive the y_hat, x_hat, and z_hat
variables that are needed, but I don't see any code in ms_var that derive
these variables. Can you more specifically point out where the code is that
shows the derivation of these matrices?

Separate from those questions I am wondering what kind of bias is
introduced if I use the filtered probabilities from ms_var? Could I use
those instead of attempting to predict with data set A and evaluate with
data set B. The reason I like the two data set methodology is that the
training data (A) is separated from the evaluation data (B) so there can't
be any bias in terms of measuring how the trained data generalizes when
benchmarked on evaluation data because the training model never saw data
set (B). Chapter 23 says the filtered probabilities only use data up until
that point in time, but it uses estimates that were built from all
information that is available. It seems biased to evaluate the residuals
using filtered probabilities (or smoothed probabilities) because training
and evaluating error on the same data set seems wrong. What do you think
the right way is to use these tools to avoid bias when measuring error of
model performance?

Thanks for any information. Also is there any possibility for us to chat on
IRC? I'm 'bmbouter' in #scilab on freenode if you want to chat there. It
would probably be faster than e-mail.

Thanks!
Brian


On Thu, Feb 12, 2015 at 3:44 PM, Eric Dubois <grocer.toolbox at gmail.com>
wrote:

> Dear Brian.
>
> If I have well understood, you want:
> - to estimate a ms_var model on a subset of your dataset;
> - recover the estimated parameters;
> - and calculate the filtered state probabilities on the other part of your
> dataset with these parameters.
>
> This can be done:
> - the function MSVAR_Filt calculates among other the filetered
> probabilities (5th output);
> - the function needs among other things the parameters of the model; they
> can be recovered from the output tlist of function ms_var; if give it the
> name res (with --> res=ms_var(...)): this is the field 'coeff' in the
> output tlist (res('coeff') with this example);
>
> But the function MSVAR_Filt also has to be fed with matrices y_hat, x_hat
> and z_hat that are matrices derived from the matrix of endogenous and
> exogenous variables (see function ms_var to see how it is done).
>
> If you are not too in a hurry, I can write the function that gathers all
> these operations within a few weeks.
>
> Éric.
>
> 2015-02-12 16:56 GMT+01:00 Brian Bouterse <bmbouter at gmail.com>:
>
>> I use GROCER's ms_var function to estimate a single variable VAR model,
>> and it estimates parameters as expected and described by the manual. I want
>> to train and evaluate my model on different data sets to avoid bias from
>> training and benchmarking on the same data set. How can this be done?
>>
>> For example consider data set A (month 1) and data set B (month 2) from a
>> 2 month sample. I would like to train on month 1 and then benchmark on
>> month 2.
>>
>> I use ms_var to train on data set A. It gives me estimated parameters and
>> filtered regime probabilities. That works well. How can I use the trained
>> parameters to then estimate on month 2 data?
>>
>> I'm aware of the ms_forecast function, but it seems to only forecast
>> using the results from an estimator like ms_var(). The forecasting will
>> then only be done on the same data as was used for estimating. I want to
>> use the trained parameters to product estimates for a different data set.
>>
>> Thanks in advance. I really appreciate being able to use this software.
>>
>> -Brian
>>
>> --
>> Brian Bouterse
>>
>> _______________________________________________
>> users mailing list
>> users at lists.scilab.org
>> http://lists.scilab.org/mailman/listinfo/users
>>
>>
>
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users
>
>


-- 
Brian Bouterse
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20150217/aea3f9a1/attachment.htm>