David Miller wrote:
I understand from several previous posts that the command -sampsi- uses
an approximate large sample test on proportion. power, and sample size
calculations. Specifically, it uses the normal approximation (with
correction) as opposed to an exact test. The advice in a previous post
was that the following equalities must hold in order for sampsi to work
adequately:
n1p1>=10
n1(1-p1)>=10
n2p2>=10
n2(1-p2)>=10
(see post from [email protected] entitled st:RE: calculation of
sample size, dated 8 Oct 2004)
I am trying to use sampsi to estimate the required number of samples as
follows:
sampsi 0.4 0.46, alpha(0.05) power(0.90) onesample
Stata indicates that 711 samples are required as indicated in the Stata
output below:
sampsi 0.4 0.46, alpha(0.05) power(0.90) onesample
Estimated sample size for one-sample comparison of proportion
to hypothesized value
Test Ho: p = 0.4000, where p is the proportion in the population
Assumptions:
alpha = 0.0500 (two-sided)
power = 0.9000
alternative p = 0.4600
Estimated required sample size:
n = 711
This seems to meet the n1p1>=10 etc. requirements listed above to use
the -sampsi- command. However, I am told that the right answer using
NQuery Advisor and its exact test for single proportions is 610
observations. S-Plus gives 613 as an answer. StatXact also gives a
similar answer to NQuery Advisor and S-Plus.
Does anyone know if Stata's use of the normal approximation (with
continuity correction) is indeed what is causing the 100+ discrepancy
here? Is there is an exact test in Stata that can be used instead of
-sampsi-? And are there additional criteria beyond the n1p1>=10 etc.
criteria listed in the referenced previous post that should be checked
before using -sampsi- ?
I have used the -findit- command to see if there is an exact test
available and looked at both Roger Newson's -powercal- command described
in the most recent Stata Journal (4th Quarter 2004) as well as Al
Feiveson's article entitled "Power by Simulation" (Stata Journal, 2nd
Quarter 2002) and wasn't able to find the answer to this question. Is
the -sampncti- command appropriate here?
----------------------------------------------------------------------------
In simulations, using both Wilson's score method in -ciw- and the exact
binomial test in -bitest- (if I understand its output correctly), it seems
that Stata is on the mark and NQuery Advisor, S-Plus or StatXact are not.
(See do-file below.) A sample size of 600 give about 80 to 85% power for a
two-sided test; 700 gives around 90%. Output from the do-file below:
Sample size: 600, Power: Wilson = 0.85 Exact = 0.83
Sample size: 650, Power: Wilson = 0.88 Exact = 0.88
Sample size: 700, Power: Wilson = 0.90 Exact = 0.90
Sample size: 750, Power: Wilson = 0.91 Exact = 0.91
So:
1. It doesn't appear that Stata's use of the normal approximation accounts
for the discrepancy.
2. Yes; there is an exact test available in Stata.
3. I don't know of any additional criteria to check first, other than
whether the hypothesis pair is directional.
4. -samplncti- wouldn't be appropriate here. (But I've read where power
calculations for binomial data have been done with the Student's t test
statistic in simulations, and it appears to be pretty good for that
purpose.)
Joseph Coveney
clear
set more off
set seed `=date("2005-02-02", "ymd")'
set seed0 `=date("2005-02-02", "ymd")'
local reps = 10000
set obs `reps'
generate byte Wilson = .
generate byte Exact = .
generate float pi1 = 0.46
forvalues den = 600(50)750 {
generate int den = `den'
quietly rndbinx pi1 den
quietly compress
forvalues i = 1/`reps' {
local successes = bnlx[`i']
quietly ciwi `den' `successes'
quietly replace Wilson = (r(lb) > 0.4) in `i'
quietly bitesti `den' `successes' 0.4
quietly replace Exact = (r(p) < 0.05) in `i'
}
summarize Wilson, meanonly
local Wilson = r(mean)
summarize Exact, meanonly
local Exact = r(mean)
display
display in smcl as result "Sample size: `den', Power: Wilson = " ///
%4.2f `Wilson', "Exact = " %4.2f `Exact'
drop den bnlx
}
exit
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
