RE: st: Multiple imputation with incidental selection

[Lance Erickson <lance_erickson@byu.edu>]

Thanks for your take and the citations. They helped me feel a little more grounded in the issue.

A colleague suggested an approach I hadn't thought about - try to figure out what the distribution of these "not applicable" responses would be by looking at a comparable data set. Since my data set is nationally representative, that didn't end up being very hard. It turns out that the religiously unaffiliated engage in almost no expressions of public religious behavior (e.g., attend church) but many engage in more private behaviors like prayer (e.g., religiously unaffiliated have a mean of 2.25 on a scale from 1-4). 

Coincidentally, the results of my models don't change regardless of the method I use - multiple imputation for "not applicable" codes for religiously unaffiliated, excluding the religious unaffiliated from the model, recoding "not applicable" codes to the minimum value on the scale, and using the mean scores of the unaffiliated from a comparable dataset. I suppose this means that for the present application, I can be confident that the results aren't biased. However, I can't help but feel a little cheap about not having a more conceptually-based and sophisticated approach.

Thanks again.

Best,
Lance


-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Cameron McIntosh
Sent: Monday, May 07, 2012 8:11 PM
To: STATA LIST
Subject: RE: st: Multiple imputation with incidental selection

I agree that screening respondents out based on a "no affiliation" response may not have been the best decision on the survey developers' part, but you can't use the affiliation screener variable to impute the missing values on these, shall we say "not applicable" responses. Remember that imputation generally builds complete data regressions in order to fill in missing values. In this case your predictor (affiliation) does not exist at all for this portion of the sample with the chunk of missing data, so I'm pretty sure that you're out of luck there.  I suppose you might consider using other variables which are predictive of religious behaviours to do the imputation, but really I think there is no substitute in this case for having actually asked the questions of everyone regardless of affiliation status. I also suggest that you take a look at:
Bradlow, E.T., & Zaslavsky, A.M. (1999). A hierarchical latent variable model for ordinal data from a customer satisfaction survey with 'No Answer' Responses. Journal of the American Statistical Association, 94(445), 43-52.

Wang, L., & Fan, X. (2004). Missing Data in Disguise and Implications for Survey Data Analysis. Field Methods, 16(3), 332-351.

Reardon, S.F., & Raudenbush, S.W.  (2006). A partial independence item response model for surveys with filter questions.  Sociological Methodology, 3, 256-300.

Holman, R., Glas, C..A.W., Lindeboom, R, Zwinderman, A.H., & de Haan, R.J. (2004). Practical methods for dealing with 'not applicable' item responses in the AMC Linear Disability Score project. Health and Quality of Life Outcomes, 2(29).http://www..pubmedcentral.nih.gov/picrender.fcgi?artid=441407&blobtype=pdf

Cam    

> From: lance_erickson@byu.edu
> To: statalist@hsphsun2.harvard.edu
> Subject: st: Multiple imputation with incidental selection
> Date: Mon, 7 May 2012 18:47:45 +0000
> 
> Dear Statalisters,
> 
> My question is first conceptual and second about application in Stata.
> 
> First, is it reasonable and/or defensible to use multiple imputation for missing data due to incidental selection? 
> 
> The particulars: I have a dataset with a module on religion that asks a variety of questions about behaviors (e.g., frequency of church attendance, prayer, etc.). The first question in the module asked about religious affiliation and respondents were skipped out of the module from there if they reported no affiliation. However, just because someone isn't officially affiliated with a religion doesn't mean that they never pray, read sacred texts, etc. So, recoding unaffiliated to "never pray" seems like it would introduce some bias. On the other hand, it seems safe to assume that, on average, the unaffiliated might have lower levels of these religious behaviors than the affiliated. My (perhaps incorrect) understanding of missing data theory is that as long as I include religious affiliation in the imputation model, I can be confident that my results are unbiased. But I have a vague sense that if I impute missing values on these variables for the unaffiliated that I will also!
  i!
>  ntroduce bias. Any ideas?
> 
> Disclaimer: I've estimated 3 models (dependent variable educational attainment) using different approaches to identify any implications for my results. One model imputes values for religious behaviors (e.g., frequency of prayer) for the unaffiliated. Another model has religious behaviors for the unaffiliated coded to the minimum of the scale (i.e., never). The final model excludes the unaffiliated. I take this as evidence that the results are robust to the approach taken, but I wonder if there is a more general, conceptual approach to the issue that I haven't been able to locate.
> 
> Second, other than include religious affiliation in the imputation model, would there be a particular way to program -mi impute- to account for the incidental selection here?
> 
> Thanks,
> Lance
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
 		 	   		  
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/