Re: st: ARIMA models - determining how much final model predicts actual trend

[Nick Cox <njcoxstata@gmail.com>]

I don't know what the "observed trend" means independently of what is
predicted by a time series model. You can construct any descriptive
statistic you like based on e.g. correlation between observed and
predicted, or its square, but I'd surround that with cautions. Some or
all of this may naturally be obvious to you, but here goes.

1. -arima- in Stata is estimated by maximum likelihood. So, on a
purist view no statistic has strong meaning unless it is equivalent
one-to-one to the likelihood.

2. At a wild guess you want something akin to what is otherwise more
familiar to you, from say multiple regression or analysis of variance.
But in those cases something like R-square has some meaning in so far
as (e.g.) least squares can be interpreted as maximising R-square. See
#1 above, again.

3. These single-number measures tend to be used as propaganda weapons
(Look at how good my model is!) whereas tougher-minded evaluations
tend to be based on whether a model fitted to part of a time series
successfully predicts the rest of the data, whether the fitted model
matches independently posited ideas about the generating process, and
so on.

4. Standard technical and philosophical reservations about whether
curve-fitting amounts to any kind of "explanation".

5. No single-number measure can capture much pattern, e.g. in what the
model misses, whether the fit is equally good or poor at all times,
etc.

6. This can stand for plenty of other reservations that just don't
happen to spring to mind right now.

That said, comparing a model with covariates with one without sounds
like very good advice to me.

Nick

On Tue, Jun 19, 2012 at 10:22 PM, Kerri Coomber
<Kerri.Coomber@cancervic.org.au> wrote:

> I have been conducting a time series analysis using an arima model. I have specified my final model and would now like to determine how much this model (and therefore the covariates in the model) predict the actual downward trend in the data.  Basically I would like to be able to state that "Using the final model to predict the outcome variable we determined that it explained XX amount of change in the outcome, or XX% of the observed trend"
>
> I have attempted this by using the predict y command.  I then calculated the difference in the predicted outcome (between the start and end if the series) and compared this to the actual change.
>
> It has also been suggested to me that I compare the predicted trend using the full model to the predicted trend using a model that does not have the covariates.
>
> Has anyone conducted such an analysis, or have any suggestions as to what may be the most appropriate approach?

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/