Richard--
I agree that anything that can be done with -nestreg- can be done by
hand by putting various specifications side by side in a table--either
approach is fine by me as long as the whole table is reported. If
only one of the specifications is reported, and if selection on
F-tests is at work, it is an invalid procedure IMHO, and it is this
*predicted* use that makes me uneasy.
The parallel is also there for -stepwise- in some sense. If you ran
2^k specifications on k possible RHS variables, and reported one that
you liked best, this is -stepwise- done by hand, right? We have all
been guilty of some kind of "publication bias writ small" at some
point, I suspect, but a package that makes it easy to commit such sins
is problematic.
It is the table of -hireg- or -nestreg- output, at the bottom, showing
the model comparisons that makes me particularly uneasy. My argument
is just "don't do stepwise, in whatever form" not "don't run more than
one regression," to be clear (this is same argument that you present
as "people shouldn't just cherry-pick" below, I think). So I don't
think it rests on false premises, but perhaps I did not make myself
clear in the original post--though if I really thought no one should
run -hireg- I would not have tried to provide a working version of it!
Whether there is "nothing about nestreg that makes it more likely or
less likely that you're only going to get selective presentations of
results" is an empirical question I suppose--but I did want to draw
the poster's attention to the potential pitfall there...
On 5/18/07, Richard Williams <[email protected]> wrote:
I think there are some false premises in your argument. Sure, people
shouldn't just cherry-pick their results, doing dozens of runs and
only presenting the ones that came out significant. But heck, people
were doing that long before anybody ever thought of nestreg. There
is nothing about nestreg that makes it more likely or less likely
that you're only going to get selective presentations of results.
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
