RE: st: Accounting for measurement error in regression

["Jason Becker" <Jason.Becker@ride.ri.gov>]

Thanks Stas for this response.  This helps clarify things a bit for me-- I've been told this was measurement error but that didn't match with what I recalled and I knew I had seen the formula before.

This clears up a bunch of confusion I had based on conversations of other folks around the Department here.
_____
Jason Becker
Research Specialist
Office of Data Analysis and Research
Rhode Island Department of Education
255 Westminster Street Providence, RI 02903
(401)-222-8495

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Stas Kolenikov
Sent: Wednesday, October 20, 2010 5:06 PM
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: Accounting for measurement error in regression

On Wed, Oct 20, 2010 at 3:47 PM, Jason Becker <Jason.Becker@ride.ri.gov> wrote:
> Hello,
>
> My data has measurement error which is generally modeled as following a
> Bernoulli distribution.  The data are percentages of students at a
> school who score above a cutoff point on an exam, and the error is
> modeled as sqrt((p)*(q)/N) where p = percentage of students above the
> cutoff, q = percentage of students below the cutoff, and N is the number
> of students).

p*(1-p)/N is the sampling variability: you believe there is a true
probability of something equal to p, and out of N sampled objects, you
will observe variance p*(1-p)*N in the number of positive responses.
You probably want to address the inaccuracy of the instrument, and
that is a far more complex thing to do.

Suppose statalist subscription were only open to people with IQ above
120, and there was an instrument that gives you 3 point margin of
accuracy (standard deviation of the scores in repeated testing, or
between people of identical ability). Then for a person with a given
IQ the probability of getting the highly sought statalist subscription
is p(IQ)=Prob[ N(IQ,3) > 120 ]. To quantify the overall measurement
error for a given university, you would want to sum all
p(IQ)*(1-p(IQ)) over the Stata users in this university; this should
give you something like a variance of the number of people eligible
for subscription. Note that this is conditional on the test score, so
I imagine that for tests with good psychometric properties
(reliability), this variance will be much smaller than p*(1-p)*N where
p = Prob[ N(100,15)>120 ], which is the measure that you think of
using.

-- 
Stas Kolenikov, also found at http://stas.kolenikov.name
Small print: I use this email account for mailing lists only.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/