Hi Adrian,
The factors (or components if you used PCA) are orthogonal if you preserve orthogonality through, for example, varimax rotation. In this case, the individual regression coefficients should not change in a linear regression of some distal outcome on the factors, as you omit factor (but they will change in a logit or probit regression which imposes identificaton constraints on the error term.
If you used an oblique rotation (such as promax or direct oblimin, which may have been the default), then there will probably be some correlation between factors which will make the coefficients change as you alter the number of predictor factors in the regression.
I would say that if your factors are actually correlated in the real world (e.g., anxiety, depression, and life satisfaction) you should use oblique rotation, and also include them all in the regression to avoid omitted variables bias. Have a look at:
Conway, J.M., & Huffcut, A.I. (2003). A review and evaluation of exploratory factor analysis practices in organizational research.
Organizational Research Methods, 6(2), 147-168.
http://www.cob.unt.edu/slides/Paswan/BUSI6280/Conway_Huffcutt.pdf
Preacher, K.J., & Maccallum, R.C. (2003). Repairing Tom Swift's electric factor analysis machine. Understanding Statistics, 2(1), 13-43.
http://www.helsinki.fi/~komulain/Tilastokirjat/Repairing_Tom_Swift.pdf
Costello, A.B., & Osborne, J.W. (2005). Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most From Your Analysis. Practical Assessment, Research, and Evaluation, 10(7).
http://pareonline.net/pdf/v10n7.pdf
Instrumenting the factors might also be a good idea.
Cam
----------------------------------------
> From: [email protected]
> To: [email protected]
> Subject: st: Aren't distinct factors from factor analysis or PCA orthogonal to each other?
> Date: Mon, 17 Aug 2009 00:46:34 -0400
>
> Hello, guys.
>
> I am new to factor analysis and I am trying to use it to decompose a big matrix of economic, financial, and political variables for many countries. So I run
>
> factor var1-var100
>
> and then I look that the first 3 factors explain most of the variation in that matrix, so then I want to use these three factors to see whether another variable (a measure of the "risk" of the country) is explained by these three factors. So I do:
>
> predict factor1 factor2 factor3
> reg risk factor1 factor2 factor3
>
> and I obtain very strongly significant estimates.
>
> My first question is: if I understand correctly, these factors should be orthogonal from each other. If that's the case, a regression such as:
>
> reg risk factor1
>
> should NOT give me a different coefficient for factor1 compared to the factor1 coefficient in the first regression that includes all three factors, right? This is because if I omit factor2 and factor3, these things would go into the error term of my regression, but they wouldn't be adding any correlation between the error term and factor1, so the factor1 coefficient shouldn't change.
>
> Or should I?
>
> In my case, it does. Why is this, can you please help me understand?
>
>
>
> My second question is: I have tried to sign all of my variables, var1-var100, so that an increase in each variable represents an improvement in economic conditions.
>
> In that case, if my first regression shows negative estimates for factor1, 2, and 3, would it be fair to interpret these coefficients as "an improvement in economic conditions reduces the risk?"
>
> Thanks a lot.
> Adrian
>
>
>
>
> _________________________________________________________________
> Get back to school stuff for them and cashback for you.
> http://www.bing.com/cashback?form=MSHYCB&publ=WLHMTAG&crea=TEXT_MSHYCB_BackToSchool_Cashback_BTSCashback_1x1
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
_________________________________________________________________
Stay on top of things, check email from other accounts!
http://go.microsoft.com/?linkid=9671355
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
