RE: st: coefficient interpretation in OLS

[Pradipto Banerjee <pradipto.banerjee@adainvestments.com>]

There is an interesting interpretation of coefficients for the multivariate case from Asset Pricing Finance. Consider,  r = X b + e, where X = [f_1 f_2 f_3 ... f_n] are the independent (correlated) variables and r is the dependent variable

The solution for b is [X*inv(X'X)]'r . Consider Y = X*inv(X'X). Y can be interpreted as a "purified"/"orthogonal" version of X. More specifically, let Y =[g_1 g_2 g_3 ... g_n], then g_1 is orthogonal to f_2, f_3, ..., f_n, and unit exposure to f_1 (g_1'f_1 = 1). Each g_i can be considered as a small deviation from each correspondingly f_i to make it orthogonal to the other f_i's. To get g_1, we could have equivalently solved: max (g_1 - f_1)'(g_1 - f_1), s.t. g_1'f_1 = 1, g_1'f_j = 0 for j<>1.

Thus, the coefficients b_1, b_2, ..., b_n, can be interpreted (using b = Y'r, or b_1 = g_1'r, ..., b_n = g_n'r), as the average exposure to "r" for a "purified" (or "orthogonal") version of x_1, x_2, ..., x_n.


-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Lucas
Sent: Monday, August 20, 2012 11:33 AM
To: statalist@hsphsun2.harvard.edu
Cc: Sam Lucas
Subject: Re: st: coefficient interpretation in OLS

The funny thing is, Speed has a technically-sophisticated explanation, yet
seems to bungle the basic fact that regression is not in general about
change in X.  One cannot make inferences about change on the basis of
comparisons in the cross-section.  As the validity of such inferences
depends in part on data, it is just wrong to say regression coefficients
indicate the effect of *change* in X, though it would be okay to say the
coefficients reference the average *difference* in Y per unit *difference*
in X while conveying whatever other technical issues need be affirmed. (I
use surrounding asterisks to indicate italics).

Otherwise, if we were to regress father's current height on son's current
height we could then claim that as son's height grows, their much older
fathers (magically?) change grow in height as well.

Sam

On Sun, Aug 19, 2012 at 12:38 PM, David Hoaglin <dchoaglin@gmail.com> wrote:
>
> Clive,
>
> I guess it depends on what constitutes a waffle.  Audiences vary in
> their understanding of regression, and the challenge is to communicate
> in a way that is both technically accurate and comprehensible.  The
> wording in Terry Speed's column, which I also learned from John W.
> Tukey, is technically correct. It need not be suitable for all
> audiences.  Presenter discretion advised.
>
> Any explanation that does not involve the blunder of "with the other
> variables held constant" is a big improvement.
>
> David Hoaglin
>
> On Sun, Aug 19, 2012 at 2:18 PM, Clive Nicholas
> <clivelists@googlemail.com> wrote:
> > David Hoaglin replied:
> >
> >> Since the definition of a coefficient in a multiple regression
> >> involves the set of other predictors in the model, Lynn should report
> >> those other variables, whose contributions are being adjusted for.
> >>
> >> No "utter waffle" is involved; the proof is straightforward
> >> mathematics.  It would be nice if multiple regression were simpler,
> >> but it is not.  The distortion comes in using the oversimplified
> >> interpretation "with the other variables held constant."  I have no
> >> reluctance to give an audience the longer interpretation, because that
> >> is what multiple regression actually does.  Better that than to
> >> deceive.  One can often dispense with "in the data at hand"; and
> >> instead of "allowing for simultaneous linear change in", one can say
> >> "adjusting for the contributions of" (as I did in my reply to Lynn).
> >> It would mislead some audiences to say "controlling for" instead of
> >> "adjusting for".
> >
> > Well, it sounds like waffle to me and I stand by it; you haven't
> > actually said whether you have used Speed's description, word for
> > word, to an audience before. I've already said I haven't and I
> > wouldn't. Alternatively - partly quoting you - I'd see nothing wrong
> > in saying "X's effect on Y is positive and significant, adjusting for
> > contributions made by the other variables in the model" to an
> > audience, and it's a damned sight less waffly than the explanation
> > offered by Speed. It's my opinion, and you don't have to buy it.
> >
> > --
> > Clive Nicholas
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

 This communication is for informational purposes only. It is not intended to be, nor should it be construed or used as, financial, legal, tax or investment advice or an offer to sell, or a solicitation of any offer to buy, an interest in any fund advised by Ada Investment Management LP, the Investment advisor.  Any offer or solicitation of an investment in any of the Funds may be made only by delivery of such Funds confidential offering materials to authorized prospective investors.  An investment in any of the Funds is not suitable for all investors.  No representation is made that the Funds will or are likely to achieve their objectives, or that any investor will or is likely to achieve results comparable to those shown, or will make any profit at all or will be able to avoid incurring substantial losses.  Performance results are net of applicable fees, are unaudited and reflect reinvestment of income and profits.  Past performance is no guarantee of future results. All f!
 inancial data and other information are not warranted as to completeness or accuracy and are subject to change without notice.

Any comments or statements made herein do not necessarily reflect those of Ada Investment Management LP and its affiliates. This transmission may contain information that is confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is strictly prohibited. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/