1 item has been added to your cart.
| Title | Negative and missing R-squared for 2SLS/IV | |
| Authors | William Sribney, Vince Wiggins, and David Drukker, StataCorp |
Two-stage least-squares (2SLS) estimates, or instrumental variables (IV) estimates, are obtained in Stata using the ivregress command.
ivregress sometimes reports no R2 and returns a negative value for the model sum of squares in e(mss).
Three-stage least-squares (3SLS) estimates are obtained using reg3. reg3 sometimes reports a negative R2 and model sum of squares. The discussion below focuses on 2SLS/IV; the issues for 3SLS are the same.
Missing R2s, negative R2s, and negative model sum of squares are all the same issue.
Stata’s ivregress command suppresses the printing of an R2 on 2SLS/IV if the R2 is negative, which is to say, if the model sum of squares is negative.
Whether a negative R2 should be reported or simply suppressed is a matter of taste. At any rate, the R2 really has no statistical meaning in the context of 2SLS/IV.
If it makes you feel better, you can compute the R2 yourself from the returned results (see An example section of the FAQ).
For two-stage least squares, some of the regressors enter the model as instruments when the parameters are estimated. However, since our goal is to estimate the structural model, the actual values, not the instruments for the endogenous right-hand-side variables, are used to determine the model sum of squares (MSS). The model’s residuals are computed over a set of regressors different from those used to fit the model. This means a constant-only model of the dependent variable is not nested within the two-stage least-squares model, even though the two-stage model estimates an intercept, and the residual sum of squares (RSS) is no longer constrained to be smaller than the total sum of squares (TSS). When RSS exceeds TSS, the MSS and the R2 will be negative.
The formula for R-squared is
R2 = MSS/TSS
where
MSS = model sum of squares = TSS − RSS and
TSS = total sum of squares = sum of (y − ybar)2 and
RSS = residual (error) sum of squares = sum of (y − Xb)2
For your model, MSS is negative, so R2 would be negative.
MSS is negative because RSS is greater than TSS. RSS is greater than TSS because ybar is a better predictor of y (in the sum-of-squares sense) than Xb!
How can Xb be worse than ybar, especially when the model includes the constant term? At first glance, this seems impossible. But it is possible with the 2SLS/IV model.
Here are the background essentials:
Let Z be the matrix of instruments (say, z1, z2, z3, z4).
Let X be the matrix of regressors (say, y2, y3, z3, z4, where y2 and y3 are endogenous and z3 and z4 are exogenous).
Let y be the endogenous variable of interest. That is, we want to estimate b, where
y = Xb + error
Let P = Z (Z'Z)−1 Z' be the projection matrix into the space spanned by Z.
2SLS/IV gives point estimates
b = ((PX)' PX)-1 (PX)' y
The coefficients are simply those from an ordinary regression but with the predictors in the columns of PX (the projection of X into Z space).
Let’s assume you have two endogenous right-hand-side variables (y1 and y2), two exogenous variables (x1 and x2), and two instruments not in the structural equation (z1 and z2). This makes your structural equation
y = (Y)B1 + (X)B2 + e
or
y = b1*y1 + b2*y2 + b3*x1 + b3*x2 + e
(where B1 and B2 are components of the vector of coefficients—b). If you run the following,
. regress y1 x1 x2 z1 z2 . predict yhat1 . regress y2 x1 x2 z1 z2 . predict yhat2 . regress y yhat1 yhat2 x1 x2
you will get exactly the coefficients of the 2SLS/IV model (but you will get different standard errors):
. ivregress 2sls y (y1 y2 = z1 z2) x1 x2
Now if we computed residuals after
. regress y yhat1 yhat2 x1 x2
the residuals would be
r = y − (PX)b
The sum of squares of these residuals would always be less than the total sum of squares.
But these are not the right residuals for 2SLS/IV. Because we are fitting a structural model, we are interested in the residuals using the actual values of the endogenous variables.
The correct two-stage least-squares residuals are
e = y − Xb
Here there is no guarantee that the sum of these residuals squared are less than the total sum of squares. These residuals do not come from a model that nests a constant-only model of y.
Let’s take a simple, and admittedly silly, example from our favorite dataset—auto.dta.
. sysuse auto, clear
(1978 Automobile Data)
. ivregress 2sls price (mpg = foreign) headroom
Instrumental variables 2SLS regression Number of obs = 74
Wald chi2(2) = 1.15
Prob > chi2 = 0.5619
R-squared = .
Root MSE = 3363.6
| price | Coef. Std. Err. z P>|z| [95% Conf. Interval] | |
| mpg | 154.4941 239.2968 0.65 0.519 -314.519 623.5072 | |
| headroom | 836.4137 821.6528 1.02 0.309 -773.9962 2446.824 | |
| _cons | 371.36 7268.765 0.05 0.959 -13875.16 14617.88 | |
There is your negative model sum of squares (−202135715). The model sum of squares is just the improvement over the sum of squares about the mean given by the full model. In this example, the sum of squared residuals from the model predictions is 837201111, whereas the sum of squared residuals about the mean of price is 635065396. By computing the model sum of square as
. display "MSS: " %15.0f 635065396 - 837201111 MSS: -202135715
we can see that our model actually performs worse than the mean of price. Why didn’t our constant keep this from happening? The coefficients are estimated using an instrument for mpg. Thus the constant need not provide an intercept that minimizes the sum of squared residuals when the actual values of the endogenous variables are used.
Just to be sure, let’s perform the sum of square computations by hand.
To get the sum of squared residuals for our model, type
. predict double errs, residuals . gen double errs2 = errs*errs . summarize errs2
| Variable | Obs Mean Std. Dev. Min Max | |
| errs2 | 74 1.13e+07 2.01e+07 3017.3 9.57e+07 |
which agrees exactly with the returned results from ivregress.
. display "ESS: " %15.0f e(rss) ESS: 837201111
To get the total sum of squared residuals about the mean of price, type
. summarize price
| Variable | Obs Mean Std. Dev. Min Max | |
| price | 74 6165.257 2949.496 3291 15906 |
| Variable | Obs Mean Std. Dev. Min Max | |
| pbarErr2 | 74 8581965 1.69e+07 .065924 9.49e+07 |
So, our “hand” computations also give a model sum of squares of −202135715 and agree with the value returned by ivregress.
What does it mean when RSS is greater than TSS? Does this mean our parameter estimates are no good? Not really. You can easily develop simulations where the parameter estimates from two-stage are quite good while the MSS is negative. Remember why we fit two-stage models. We are interested in the parameters of the structural equation—the elasticity of demand, the marginal propensity to consume, etc. If our two-stage model produces estimates of these parameters with acceptable standard errors, we should be happy—regardless of MSS or R2. If we were interested strictly in projections of the dependent variable, we should probably consider the reduced form of the model.
Another way of stating this point is that there are models in which in the distribution of 2SLS estimates of the parameters will be well approximated by its theoretical distribution but that the R2 computed from some samples will be negative. There are several ways of illustrating this point. Perhaps the most accessible is via simulation.
We simulate data from the model
(1) y = 1 + − .1*x + e1 + e2
(2) x = w + z + c1 + .5*e1
(3) z = 1.5*c1 + e3
where e1, e2, w, and c1 are all independent normal random variables. The c1 term in (2) and (3) provide the correlation between x and z. The e1 term in (1) and (2) is the source of the correlation between x and the error term (e1 + e2) for y. The coefficient of −0.1 is the parameter that we are trying to estimate. We are going to estimate this parameter with 2SLS using ivregress with y as the dependent variable, x as the endogenous variable, and z as the instrument for x. For each simulated sample, we construct y, x, and z using independent draws of the standard normal variables e1, e2, w, and c1 and (1)–(3). Then we use
. ivregress 2sls y (x = z)
to estimate the coefficient −0.1. For each simulated sample, we record the following statistics:
| b1 | estimate of the coefficient (−.1) |
| p | p of the null hypothesis that b1 = −.1 |
| reject | if p<.05 and 0 otherwise |
| r2 | computed R2 (missing if mss < 0) |
| mss | value of the model sum of squares |
| rho_x1e | correlation between x1 and e=e1+e2 |
| rho_x1z1 | correlation between x1 and z1 |
| fsf | first stage F statistic |
| p_fsf | p-value from the first stage F statistic |
The Stata code for drawing 2,000 simulations of this model, estimating the coefficient −0.1, computing the statistics of interest, and finally, summarizing the results, is saved in the file negr2.do. Each simulated sample contains 1,000 observations, so the results should not be attributed to a small sample size.
Here are the results we obtained with the summarize command:
. summarize
| Variable | Obs Mean Std. Dev. Min Max | |
| b1 | 2000 -.0981982 .0541345 -.2771809 .0765793 | |
| p | 2000 .4945649 .2884685 .0002706 .9995125 | |
| reject | 2000 .0485 .214874 0 1 | |
| r2 | 64 .0068443 .0063426 .000051 .0264567 | |
| mss | 2000 -78.4407 49.08486 -273.4773 47.94914 | |
| rho_x1e | 2000 .235859 .0300348 .1194255 .3460462 | |
| rho_x1z1 | 2000 .5556971 .0216154 .4764362 .6183904 | |
| fsf | 2000 448.584 50.32493 293.0595 617.9501 | |
| p_fsf | 2000 2.62e-34 7.49e-33 0 3.29e-31 |
The results for rho_x1e, rho_x1z1, fsf, and p_fsf indicate the correlations between the endogenous variable and the error term and between the endogenous variable and its instrument are reasonable and there is no weak-instrument problem. The results for b1, p, and reject indicate that the mean estimate of the coefficient on x is very close to its true value of −0.1 and that there is no size distortion of the test that coefficient on x = −0.1. In short, the distribution of the estimates, b1, is very well approximated by its theoretical asymptotic distribution. Together, these results imply that the 2SLS estimator is performing according to the theory in these simulations.
There are only 64 observations on r2 because there are 1,936 observations in which mss < 0 for the result of
. count if mss < 0 1,936
Thus the results illustrate that there is at least one model for which the distribution of the 2SLS estimates of the parameters is very well approximated by its asymptotic distribution but that the R2 will be negative in most of the individual samples. To obtain more models that produce the same qualitative results, simply change the coefficient −0.1 by a small amount. As one would expect, increasing the coefficient −0.1 reduces the fraction of the of simulated samples that produce a negative R2.
