[Scilab-users] Using GROCER ms_var parameters for forecasting

[Eric Dubois]

Dear Brian.

1) sorry, I made indeed a typo and wanted to speak about y_mat, x_mat and
z_mat.

2) I do not know exactly what you want, but you can calculate what you want
from the parameters and all other inputs

3) you will find attached a function run_ms_var that performs, I hope, what
you need: this function takes a results tlist from a ms_var execution and a
vector of endogenous variables to feed the VAR (your benchmark data).

I have checked that if you give as endogenous variables exactly the same
variables as the one used for estimation, you recover the same yhat,
filtered probs, etc.

To use the function, you have to save it in a folder, say c:/newms, and run
into Scilab
--> getd('c:/newms)

To check what I mentionned above, run:
--> load(GROCERDIR+'\data\us_revu.dat')
--> bounds('1967m4','2004m2')
--> nb_states=2
--> switch_var=2 // variances are switching
--> var_opt=3 // heteroskedastik var-cov matrix
--> r=ms_var('cte',3,'100*(log(us_revu)-lagts(2,log(us_revu)))',nb_states,switch_var,var_opt,'prt=initial;final','transf=stud')
--> [y_hat,resid,PR,PR_STT,PR_STL]=run_ms_var(r,'100*(log(us_revu)-lagts(2,log(us_revu)))'
--> PR_STT-r('filtered probs')

The function is rather rough (no header, no options,...) and can be
improved, but I hope it answers your needs.

Éric.




2015-02-17 15:03 GMT+01:00 Brian Bouterse <bmbouter at gmail.com>:

> Hi Eric,
>
> Thanks for the reply! Yes you understand my goals correctly, but one
> clarification: It would be better to have the estimated values directly
> instead of the filtered state probabilities. I usually get these with
> ms_forecast(r, n).
>
> I've been reading through the grocer code to determine how to write the
> function you suggest. I do need it sooner than a few weeks so I'm
> attempting to do it. It seems straightforward except for the y_hat, x_hat,
> and z_hat variables I need to provide to MSVAR_Filt.(). Here are some
> questions:
>
> 1) You say I need to feed MSVAR_Filt() with y_hat, x_hat, and z_hat, but
> the variables in the function signature for MSVAR_Filt read
> as y_mat,x_mat,z_mat. Did you mean y_mat or y_hat?
>
> 2) y_hat (2nd output) is an output of MSVAR_Filt(). The function comments
> say that is my estimated y. Is that the direct estimates that I am looking
> for?
>
> 3) I read through ms_var() to see how to derive the y_hat, x_hat, and
> z_hat variables that are needed, but I don't see any code in ms_var that
> derive these variables. Can you more specifically point out where the code
> is that shows the derivation of these matrices?
>
> Separate from those questions I am wondering what kind of bias is
> introduced if I use the filtered probabilities from ms_var? Could I use
> those instead of attempting to predict with data set A and evaluate with
> data set B. The reason I like the two data set methodology is that the
> training data (A) is separated from the evaluation data (B) so there can't
> be any bias in terms of measuring how the trained data generalizes when
> benchmarked on evaluation data because the training model never saw data
> set (B). Chapter 23 says the filtered probabilities only use data up until
> that point in time, but it uses estimates that were built from all
> information that is available. It seems biased to evaluate the residuals
> using filtered probabilities (or smoothed probabilities) because training
> and evaluating error on the same data set seems wrong. What do you think
> the right way is to use these tools to avoid bias when measuring error of
> model performance?
>
> Thanks for any information. Also is there any possibility for us to chat
> on IRC? I'm 'bmbouter' in #scilab on freenode if you want to chat there. It
> would probably be faster than e-mail.
>
> Thanks!
> Brian
>
>
> On Thu, Feb 12, 2015 at 3:44 PM, Eric Dubois <grocer.toolbox at gmail.com>
> wrote:
>
>> Dear Brian.
>>
>> If I have well understood, you want:
>> - to estimate a ms_var model on a subset of your dataset;
>> - recover the estimated parameters;
>> - and calculate the filtered state probabilities on the other part of
>> your dataset with these parameters.
>>
>> This can be done:
>> - the function MSVAR_Filt calculates among other the filetered
>> probabilities (5th output);
>> - the function needs among other things the parameters of the model; they
>> can be recovered from the output tlist of function ms_var; if give it the
>> name res (with --> res=ms_var(...)): this is the field 'coeff' in the
>> output tlist (res('coeff') with this example);
>>
>> But the function MSVAR_Filt also has to be fed with matrices y_hat, x_hat
>> and z_hat that are matrices derived from the matrix of endogenous and
>> exogenous variables (see function ms_var to see how it is done).
>>
>> If you are not too in a hurry, I can write the function that gathers all
>> these operations within a few weeks.
>>
>> Éric.
>>
>> 2015-02-12 16:56 GMT+01:00 Brian Bouterse <bmbouter at gmail.com>:
>>
>>> I use GROCER's ms_var function to estimate a single variable VAR model,
>>> and it estimates parameters as expected and described by the manual. I want
>>> to train and evaluate my model on different data sets to avoid bias from
>>> training and benchmarking on the same data set. How can this be done?
>>>
>>> For example consider data set A (month 1) and data set B (month 2) from
>>> a 2 month sample. I would like to train on month 1 and then benchmark on
>>> month 2.
>>>
>>> I use ms_var to train on data set A. It gives me estimated parameters
>>> and filtered regime probabilities. That works well. How can I use the
>>> trained parameters to then estimate on month 2 data?
>>>
>>> I'm aware of the ms_forecast function, but it seems to only forecast
>>> using the results from an estimator like ms_var(). The forecasting will
>>> then only be done on the same data as was used for estimating. I want to
>>> use the trained parameters to product estimates for a different data set.
>>>
>>> Thanks in advance. I really appreciate being able to use this software.
>>>
>>> -Brian
>>>
>>> --
>>> Brian Bouterse
>>>
>>> _______________________________________________
>>> users mailing list
>>> users at lists.scilab.org
>>> http://lists.scilab.org/mailman/listinfo/users
>>>
>>>
>>
>> _______________________________________________
>> users mailing list
>> users at lists.scilab.org
>> http://lists.scilab.org/mailman/listinfo/users
>>
>>
>
>
> --
> Brian Bouterse
>
> _______________________________________________
> users mailing list
> users at lists.scilab.org
> http://lists.scilab.org/mailman/listinfo/users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/users/attachments/20150217/61ed874f/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: run_ms_var.sci
Type: application/octet-stream
Size: 2600 bytes
Desc: not available
URL: <https://lists.scilab.org/pipermail/users/attachments/20150217/61ed874f/attachment.obj>