RE: Re: st: Subgroup analysis

["Lachenbruch, Peter" <Peter.Lachenbruch@oregonstate.edu>]

Also, if one does many tests, it is easy to find a lot of spurious 'effects.'
A recent Lancet article purported to show that eating cereal while pregnant gave a higher probability of male children.  The problem was that the authors had looked at 131 variables at 2 trimesters and picked out the variable that had the lowest p-value.  It was about 2007.  these were Epidemiologists who should know something about the way sex is determined - XX and XY etc.

Tony

Peter A. Lachenbruch
Department of Public Health
Oregon State University
Corvallis, OR 97330
Phone: 541-737-3832
FAX: 541-737-4001


-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Clyde Schechter
Sent: Thursday, July 08, 2010 6:33 AM
To: statalist@hsphsun2.harvard.edu
Subject: Re: Re: st: Subgroup analysis

Comparing the statistical significance of effects in two sub-populations
is rather perilous.

I have two suggestions.  First, since you have already done the
race-specific analyses, just look at the coefficients in the White and
African American subgroups, disregarding standard errors and p-values. 
Are the coefficients similar?  If so, you may well be simply finding a
lack of statistical power to detect in a subgroup of 600 subtle effects
that achieve statistical significance in your larger combined sample.

Second, and more formally, before even running the subgroup analyses, I
would have added race X predictor interaction terms to the model and then
tested the significance of those interaction terms.  If _they_ are not
significant, then the conclusion would be that your data do not provide
evidence of difference across races (which is not the same as evidence of
no difference across races).  If the interaction terms _are_ significant,
then the coefficients of those interaction terms give you estimates of the
cross-race differences in effects.

Hope this helps.

Clyde Schechter, MA MD
Associate Professor of Family & Social Medicine

Please note new e-mail address: clyde.schechter@einstein.yu.edu

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/