Re: st: Relative Importance of predictors in regression

[Lucas <lucaselastic@gmail.com>]

Hi David,

I am looking for the mathematical expression you indicated would make
it clear which interpretation is correct. The mathematical
manipulation isn't very helpful, because someone who interprets the
issue differently than you do before can interpret this demonstration
differently than you do. So, do either of those books have the
mathematical expression you mentioned? If so, I'll check it out.

On change vs. difference, discrete things change or do not, and
non-discrete things change or do not.  The distinction between "change
and difference" is orthogonal to the distinction between "discrete and
non-discrete."

Indeed, the analogy you deploy to support the change interpretation,
using slopes and hills, is one reason people say "held constant."  The
difference (slope) between my height on the hill and Joe's height on
the hill is distinct from (and independently estimable given) our
horizontal placement on the hill. Horizontal placement, thus, is "held
constant." If this is incorrect, it shows why analogies are less
helpful than mathematical expressions. Thus, my request for the
mathematical expression you indicated was available.

I do not understand why you retain one mis-interpretation of the
regression model that is extremely elementary and easily corrected,
but are adamant that everyone else is wrong if they use (what you
call) another mis-interpretation of the model, a mis-interpretation
that 1)can be shown with straightforward mathematical expressions but
then 2)seems so complex that it cannot be written in plain text.

Anyway, please let me know which of those textbooks have the
mathematical expression you referenced earlier.  I'll pull it from the
library and take a look

Thanks!

Sam

On Tue, Nov 5, 2013 at 5:25 AM, David Hoaglin <dchoaglin@gmail.com> wrote:
> Hi, Sam.
>
> No "wordplay" is involved.  Regression coefficients are a type of
> slope, so it is natural to talk about "increase" and "change," which
> most people would consider to be continuous, whereas "difference"
> often comes across as discrete.
>
> Earlier, when I mentioned subspaces and projections, I had in mind a
> description based on linear algebra.  Since you teach regression, you
> should be familiar with partial-regression plots (also called
> added-variable plots), and I can use that setting to describe the main
> idea.
>
> Suppose Y = b1X1 + b2X2 + ... + bpXp.  The observed values of Y
> usually include disturbances (or noise or errors), but I am using the
> true values of Y in order to describe the interpretation of the true
> regression coefficients.  The description applies with minor
> modifications when Y includes noise.  I have not numbered the
> predictors from 0, so X1 will often be the constant predictor, and I
> will focus on b2.
>
> Regress Y on X1, X3, ..., Xp, producing the fitted regression c1X1 +
> c3X3 + ... + cpXp and the residuals Y - (c1X1 + c3X3 + ... + cpXp).
> This step adjusts Y by removing the contributions of X1, X3, ..., Xp.
>
> Regress X2 on X1, X3, ..., Xp, producing the fitted regression a1X1 +
> a3X3 + ... + apXp and the residuals X2 - (a1X1 + a3X3 + ... + apXp).
> This step adjusts X2 for simultaneous linear change in X1, X3, ...,
> Xp.
>
> Regress the Y-residuals on the X2-residuals.  The resulting regression
> line (through the origin) is
> Y - (c1X1 + c3X3 + ... + cpXp) = b2[X2 - (a1X1 + a3X3 + ... + apXp)].
> That is, the slope of this regression line is the coefficient of X2 in
> the original model.  And, because the values of Y do not include
> noise, the points (X2-residual, Y-residual) lie on that line through
> the origin.
>
> I hope this explanation will help you to explain the correct general
> interpretation in your teaching, and steer your students away from the
> "held constant" interpretation.  I realize that it may be awkward if
> the textbook uses the "held constant" interpretation (students will
> wonder what else the book gets wrong), but the message is important.
> Some books do get multiple regression right.  One example is Weisberg
> (1985).  And Cook and Weisberg (1982, Section 2.3.2) give a
> straightforward proof for the usual partial-regression plot.
>
> Regards,
>
> David Hoaglin
>
> Cook, R. D., and S. Weisberg (1982).  Residuals and Influence in
> Regression.  New York: Chapman & Hall.
>
> Weisberg, S. (1985).  Applied Linear Regression, second edition.  New
> York: John Wiley & Sons.
>
> On Mon, Nov 4, 2013 at 9:13 PM, Lucas <lucaselastic@gmail.com> wrote:
>> Well, just two things:
>>
>> 1)The word "change" means change (Xa becomes Xb); the word
>> "difference" means difference (Xa is not the same as Xb).  Why
>> wordplay?  If difference is always an accurate statement of the model,
>> and "change" is only accurate sometimes (as your response implicitly
>> admits), why not say, "Wow, my bad.  Yes, difference is generally
>> accurate, and that of course is a better way to describe the model in
>> general, and especially in a context with people from dozens of
>> fields."?
>>
>> 2)You indicated that "Straightforward mathematics" proved the point
>> you have often articulated.  Now you indicate is too difficult to
>> express in plain text, which certainly does not sound like
>> straightforward mathematics.  But, okay.  Still, I would love to see
>> this expression.  I'm not joking or baiting--I teach this material and
>> if there's an expression that would make your point I'll integrate it
>> into my teaching. Others on the list might do so as well.  That would
>> be a major service to the disciplines. Perhaps instead of trying to
>> write it in plain text, you can provide a citation to something that
>> clearly makes your point. That could be helpful.
>>
>> So, at this point I, and perhaps others, look forward to receiving
>> that citation.  Thanks in advance.
>>
>> Take care.
>> Sam
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/