I would argue that it depends what you mean by "post hoc power
calculations".
If Nico is proposing to calculate the power to detect a POPULATION odds
ratio of the size of the observed SAMPLE odds ratio, then that would
indeed be misleading and uninformative. The Central Limit ?Theorem
implies that the power to detect a POPULATION odds ratio the size of the
observed SAMPLE odds ratio, using the observed SAMPLE P-value as the
threshold, will always be 0.5 (or 50 percent). Therefore, it will not be
surprising if the study was underpowered to detect a POPULATION odds
ratio as large as the observed SAMPLE odds ratio, even though the study
may have been adequately powered to detect a larger POPULATION odds
ratio, which was considered to be clinically interesting by the study
designers at the time of the study design.
HOWEVER, if Nico intends to use an existing study to inform the design
of a possible larger (and therefore more sensitive) study, using the
existing study as a pilot, then Nico is doing nothing wrong. In fact,
Nico is probably doing the best that anybody can do, given that the
proposed analysis method appears to be confounder-adjusted, implying
that doing power calculations from first principles will be very
complicated. And there may be a genuine lack of consensus regarding what
effect size is "clinically interesting".
I hope this helps.
Best wishes
Roger
Roger Newson
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: [email protected]
Web page: www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/pop
genetics/reph/
Opinions expressed are those of the author, not of the institution.
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of uri goldbourt
Sent: 15 August 2007 20:30
To: [email protected]
Subject: RE: st: Power calculation for Beta/Odds Ratios in logistic
regression models
Well and importantly said!
UG
---------------------
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Neil Shephard
Sent: Wednesday, August 15, 2007 9:08 AM
To: [email protected]
Subject: Re: st: Power calculation for Beta/Odds Ratios in logistic
regression models
On 8/15/07, Nico Hutter <[email protected]> wrote:
> Hi everyone,
>
> we would like to run a power calculation for beta-coefficients / odds
> ratios in logistic regression models with covariates.
> Is such a procedure implemented in SATA?
>
> Used command for the logistic regression model:
> svy: logit depression age sex comorbidity, or
>
> Data stems from a national representative survey. Dependent variable
is
> depression. Comorbidity is dichotomised. Now, we are interested in the
> "post hoc" power of the odds ratio of comorbidity.
>
> Can anybody give us some advice, please? Thanks in advance!
Roger Newson has pointed to an appropriate solution, but I would
question why you wish to do this?
In my opinion "post hoc" power is a meaningless measurement that is
completely useless in interpreting results, and a practice that needs
to discouraged.
You have already done your test (logistic regression) and got your
results. Knowing the 'power' your data had of detecting this size of
effect (or more likely lack of) will tell you nothing more informative
about the association. Its like trying to tell someone who's just one
the lottery that they shouldn't buy lottery tickets because the chance
of winning is so low, they're not going to care as they've already got
their answer.
There are a few papers around that discuss this in greater detail (I'm
sure there are more).
Neil
References
Goodman SN, Berlin JA (1994) The Use of Predicted Confidence Intervals
when Planning Experiments and the Misuse of Power When Interpreting
Results. Annals of Internal Medicine 121.3:200-206
Hoenig J.M., Heisey D.M. (2001) The Abuse of Power: The Pervasive
Fallacy of Power Calculations for Data Analysis. The American
Statistician 55:19-24
Levine M, Ensom MH (2001) Post hoc power analysis: an idea whose time
has passed? Pharmacotherapy 21.4:405-409
--
"In mathematics you don't understand things. You just get used to
them." - Johann von Neumann
Email - [email protected] / [email protected]
Website - http://slack.ser.man.ac.uk/
Photos - http://www.flickr.com/photos/slackline/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
