Re: st: Question on Wooldridge's Procedure 18.1

[Austin Nichols <austinnichols@gmail.com>]

Brent Gibbons <brent.gibbons@gmail.com>:

You are misunderstanding what pure noise refers to in my post at
 http://www.stata.com/statalist/archive/2010-01/msg00144.html
I proposed a thought experiment where you generate pure noise
variables z1-z20 (say) to serve as (very bad) excluded instruments Z,
where the predicted value of an endogenous X might still be highly
correlated with observed X, leading you to think you had strong
instruments using Procedure 18.1 (naively treating the predicted X as
your sole excluded instrument). Nonlinearity of the first stage is not
required for this problem to be observed--only an incorrect
understanding of what the weak instrument tests are designed for.  No
bias is introduced by the nonlinear transformation of exogenous
variables in forming a generated instrument, however.  The bias would
come only from violating the 2 crucial assumptions required for
instruments: Z correlated with X and uncorrelated with the error (the
first assumption fails in the thought experiment). E.g.

clear
set seed 12345
drawnorm z1-z20 e, n(1000)
g x=e/4+rnormal()
g y=x+e
qui reg x z*
predict xhat
ivreg2 y (x=xhat)
ivreg2 y (x=z*)

Note how the 18.1 approach does not catch the weak instruments
problem, but using the original instruments does.

Also note that the paper I cited as an example of the flawed test for
weak instruments is now published:
 http://www.aeaweb.org/articles.php?doi=10.1257/aer.102.5.1927
mostly without discussion of weak instruments, with a note (to Table 5) that
 "F-statistics are for tests that all instruments equal zero in
first-stage equations"
where the minimum eigenvalue of the Cragg-Donald stat should be
reported (see e.g. http://www.nber.org/papers/t0284.pdf). The
instruments look weak in all specifications, by the Stock-Yogo
standard, but the paper appears in a top econ journal, so take heart.

On Wed, Aug 8, 2012 at 7:00 PM, Brent Gibbons <brent.gibbons@gmail.com> wrote:
> Hopefully the list server will recognize this response in the correct
> thread. I was trying to use nabble (and only get the digest) but it
> wasn't allowing me to respond. In case it doesn't recognize it
> correctly, the last 2 threads are below.
>
> Austin, thanks for your response. Let me see if I can explain a bit better.
>
> I was partly trying to ask why exactly you have stated weak instrument
> tests for proc. 18.1 are problematic. You mention that "weak
> instrument diagnostics should come from straight IV, not procedure
> 18.1--note that procedure 18.1 would go through if Z was pure noise,
> and the predicted value of your endogenous variable could be very
> highly correlated with the endogenous variable, leading you to think
> you had very strong instruments."
>
> As I understand the 18.1 procedure, mainly coming from Wooldridge
> (2002), you have an extra component in the predicted probabilities
> besides what is specified in the probit model, which Wooldridge says
> allows for identification in the subsequent 2SLS even if there are no
> instruments (p. 624). The extra component comes from the probit
> estimator's "nonlinear function of x" - which is what I figured you
> were referring to with 'pure noise'. Hence my first question - is it
> this reason alone that the weak instrument tests are problematic?
>
> My second question relates to a scenario where you have both strong
> instruments and this same component from above, that is also highly
> correlated with the endogenous variable. So you can test the strength
> of the instruments with the linear 2SLS. But is there any reason to
> worry that this extra component could bias the result? I don't think
> so, but I'm having trouble explaining why to myself.
>
> Thanks again for all comments, Brent
>
> Below posted Aug. 7, 2012
>
> B.Gibbons <brent.gibbons@gmail.com>:
> This question is not clear to me--the point is that weak IV
> diagnostics work fine for the linear probability model but not
> Procedure 18.1, as evidenced by a thought experiment (or simulation)
> using white noise variables as excluded instruments as in my 2010
> post. When you say "can't test the exclusion restriction" you are
> apparently confusing several tests of quality of inference in
> instrumental variables. I have no idea what you mean by "the
> non-linearity in the probit may be correlated with..." (did you mean
> some component of the error? a generalized residual?)
>
> Below posted Aug. 2, 2012
>
> Hi Austin, I'm currently using the 18.1 method in a project and have seen
> your warnings about using tests of instrument strength through the 18.1
> method.
>
> My 1st question is whether those warnings are solely because of the
> potential that the non-linearity in the probit may be correlated with the
> binary endogenous variable - and falsely show good instrument strength.
>
> 2nd - what if there is both strong correlation between the non-linearity in
> the probit AND strong instruments in the model: is there reason to worry
> about this non-linearity as having a potential bias, especially since you
> can't test the exclusion restriction for that?
>
> Thanks for any comments, Brent
>
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/