RE: st: Sampling for proportion: characteristic of interest

["Ilian, Henry (ACS)" <Henry.Ilian@dfa.state.ny.us>]

Again, thank you.

Both of your questions are hard to answer, although the first is harder. I'd like to get the smallest feasible sample size (since people have to reread and rerate cases at approximately one case per day) with an acceptable margin of error; 10% seems ok, although, if there's a standard for interrater reliability studies, I haven't found it. I haven't thought far enough ahead to consider effect size, although if there are meaningful differences between raters, I need to detect them. 


-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Stas Kolenikov
Sent: Friday, August 24, 2012 3:33 PM
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: Sampling for proportion: characteristic of interest

OK, let's start again. What is given to you (target sample size,
target margin of error, target effect size for a given power, etc.),
and what is it that you want to compute (sample size to achieve a
given accuracy; effect size that you can detect with a given power;
etc.)?

-- 
-- Stas Kolenikov  ::  http://stas.kolenikov.name
-- Senior Survey Statistician, Abt SRBI  ::  work email kolenikovs at
srbi dot com
-- Opinions stated in this email are mine only, and do not reflect the
position of my employer

On Fri, Aug 24, 2012 at 2:11 PM, Ilian, Henry (ACS)
<Henry.Ilian@dfa.state.ny.us> wrote:
> Stas, thank you.
>
> The question was: when you need to specify a proportion to calculate a sample size, the proportion is of a characteristic of interest, but when you have several characteristics of interest, all of which are important, how do you specify the proportion.
>
> Your answer would suggest that if there are yes/no questions on the instrument, then I need to specify 50% and therefore use a larger sample size. When you say you specify 10%-20% for Likert/multinomial scales, what percentage do you use for the calculation?
>
> Henry
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Stas Kolenikov
> Sent: Friday, August 24, 2012 2:58 PM
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Sampling for proportion: characteristic of interest
>
> What exactly is your question here? You have not really stated what do
> you want to determine. You don't seem to have much of a design/sample
> size issue here, if you are set on sampling a known number of cases.
>
> When I write study precision and power sections, I usually specify
> several levels of incidence -- e.g., 10%, 20% and 50%. The 50% would
> be relevant for the Yes/No answers on your instrument; the 10%-20%
> would be relevant for your Likert/multinomial scales.
>
> --
> -- Stas Kolenikov  ::  http://stas.kolenikov.name
> -- Senior Survey Statistician, Abt SRBI  ::  work email kolenikovs at
> srbi dot com
> -- Opinions stated in this email are mine only, and do not reflect the
> position of my employer
>
>
> On Fri, Aug 24, 2012 at 11:35 AM, Ilian, Henry (ACS)
> <Henry.Ilian@dfa.state.ny.us> wrote:
>> I need to take a quality-control sample for a case-reading study. A random sample of about 600 cases will be drawn and read, and I need a sample of those to be read by a second reader. The case-reading instrument has approximately 100 questions--the final number hasn't been determined yet. The response choices mostly form ordinal scales, although a few will be nominal, hence I'll be sampling for proportion.
>>
>> Everything I can find on sampling for proportion only considers the proportions for a single characteristic of interest. I have 100+ characteristics with different expected proportions. How do I approach this problem?
>>
>> Many thanks,
>>
>> Henry Ilian
>>
>>
>>
>>
>>
>> Confidentiality Notice: This e-mail communication, and any attachments, contains confidential and privileged information for the exclusive use of the recipient(s) named above. If you are not an intended recipient, or the employee or agent responsible to deliver it to an intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and delete this communication from your computer. Thank you.
>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/