st: RE: RE: Insignificant coefficient in prediction

["Mike Kim" <kalisperos@gmail.com>]

Thanks. I share with you the skepticism on significance. Let me ask this
way. Let's say a practitioner wants to find which factor has more impact on
sales to make an investment decision: store size or good location. Let's
say, a regression model tells that the effect of store size is smaller than
the effect of location but highly significant and that the effect of
location is larger but insignificant with p>0.5, for example. What should we
recommend? To invest in store size or to pick up a better location? I guess
he should invest in store size. Then, what implication does it have on
prediction using both coefficients? Are these two problems very different
ones and should not be mixed? 

Thanks again.
Mike.

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Nick Cox
Sent: Wednesday, December 01, 2010 12:12 PM
To: 'statalist@hsphsun2.harvard.edu'
Subject: st: RE: Insignificant coefficient in prediction

You get what you ask for. If you fit a model and then follow with -predict-,
then Stata fits that model. 

You have scope to hunt for models in which every coefficient is significant
at conventional levels and then -predict- with those. 

Stata lets you do that, but it remains agnostic about your prejudices about
how to work with data. 

My own view is that insisting that every coefficient be significant is a
dogma that admits many exceptions. For example, 

1. Often the search for such a model involves a trawl through many possible
models and makes the usual interpretation of significance level problematic.
Is a project a fishing expedition or does it have a clear focus on
scientific research questions? 

2. Not every researcher feels compelled to use thresholds such as .05 or
.01. My attitude to 0.049 is not distinguishable from my attitude to 0.051. 

3. Often there are scientific and/or statistical grounds for including
bundles of predictors, even if some or all don't qualify individually as
significant. A perhaps esoteric example is that a sine and a cosine term
usually belong together, even if one is not significant. A more common
example, for many readers of this list, is that a bunch of indicators
usually belong together, ditto. 

4. Often much of the point of a project is to use the same model, meaning
here specifically the same predictors, in different circumstances. Finding
out how far results are similar or different is more straightforward than if
researchers just fit a ragbag of different models and insist on everything
being significant. 

5. Significance is usually overemphasised in any case. Often the P-value is
the dodgiest part of the results, especially if there is doubt about how far
the underlying assumptions are satisfied, as there is usually is. 

I imagine that any others whose prejudices overlap with mine could easily
extend my list. 

Nick 
n.j.cox@durham.ac.uk 

Mike Kim

I have a question about prediction. Linear predictions in Stata (probably in
all software) after any regression seem to use all estimated coefficients
regardless their statistical significance. How can I understand that we use
insignificant coefficients in forecasting?

sysuse auto, clear
reg mpg weight length  // length is not significant
predict mpgh, xb

The predicted values use all coefficients including the coefficient of
'length' even if it is not significant.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/