Re: st: Analysis of diagnostic properties of imputed ordinal score

[roland andersson <rolandersson@gmail.com>]

Thank you Cameron

The score is based on 7 variables and was constructed on a previous
set of data. Three are dichotomous, one is ordinal with 4 levels, and
three are continuous variables divided into 2-3 intervals. The missing
values are mainly in one of the continuous variables.

The new set of data is used for validation, but we also include new
variables to test if the score can be improved. We have imputed 5 sets
of data. After the imputation the continuous variables were divided
into the categories according to the construction of the score, and we
calculated the sum of the scoring points for each patient. The
distribution of the patients does not vary much between the 5
imputation sets.

Our problem was how to appropriately calculate the sensitivity and
specificity from the combined sets. I think I have come up with a
solution. I simply dichotomise the score at clinically relevant cutoff
points, and use mim with the mean command to get the proportions (with
the standard error) which represent the sensitivity and specificity.

Do you have any comments so far on how we did it?

I will read the articles and may come back if there comes up more questions.

Why do you propose 20 imputations? We got almost identical results
with 5 and 20 imputations.

Best regards
Roland Andersson
Surgeon
Jönköping, Sweden


2011/10/12 Cameron McIntosh <cnm100@hotmail.com>:
> Roland,
>
> How exactly did you impute your ordinal missing values? I would suspect that you'd have to look at the variability in accuracy (sensitivity+specificity) across all imputed datasets, and construct CIs for the ROC models based on that variability (not sure if you're also using SSLRs but the approach would be similar). I'd also use more than 5 imputations -- at least 20 but perhaps even more. I would also strongly suggest having a look at the following papers.
>
> Pérez, A., Dennis, R.J., Gil, J.F., Rondón, M.A., & López, A. (2002). Use of the mean, hot deck and multiple imputation techniques to predict outcome in intensive care unit patients in Colombia. Statistics in Medicine, 21(24), 3885-3896.
>
> Long, Q., Zhang, X., & Hsu, C.-H. (2011). Nonparametric multiple imputation for receiver operating characteristics analysis when some biomarker values are missing at random. Statistics in Medicine,
> Early View.
> http://onlinelibrary.wiley.com/doi/10.1002/sim.4338/abstract;jsessionid=63E100FD9A64CCB7B6C8E6D57CA08581.d01t02
>
> Liu, D., & Zhou, X.-H. (January 21, 2011). Semiparametric Estimation of the Covariate-Specific ROC Curve in Presence of Ignorable Verification Bias. UW Biostatistics Working Paper Series. Working Paper 374. Seattle, WA: University of Washington - Seattle Campus.
> http://www.bepress.com/cgi/viewcontent.cgi?article=1213&context=uwbiostat
>
> An, Y. (2011). Empirical Likelihood Confidence Intervals for ROC Curves with Missing Data. Mathematics Theses. Paper 95.
> http://digitalarchive.gsu.edu/math_theses/95
>
> Janssen, K.J.M., Vergouwe, Y., Donders, A.R.T., Harrell, F.E., Jr., Chen, Q., Grobbee, D.E., & Moons, K.G.M. (2009). Dealing with Missing Predictor Values When Applying Clinical Prediction Models. Clinical Chemistry, 55, 994-1001.
> http://www.clinchem.org/cgi/reprint/55/5/994
> http://www.clinchem.org/cgi/content/full/clinchem.2008.115345/DC1
>
> Liu, D., & Zhou, X.-H. (2010). A model for adjusting for nonignorable verification bias in estimation of the ROC curve and its area with likelihood-based approach. Biometrics, 66(4), 1119-1128.
>
> Hope this helps,
>
> Cam
>
>> Date: Wed, 12 Oct 2011 11:16:29 +0200
>> Subject: st: Analysis of diagnostic properties of imputed ordinal score
>> From: rolandersson@gmail.com
>> To: statalist@hsphsun2.harvard.edu
>>
>> We are analysing the diagnostic properties (sens, spec) of a clinical
>> score. We had to impute some missing values for variables that are
>> included in the score. We now have 5 imputed sets of data with a
>> clinical score with values from 1 to 14 and the corresponding
>> distribution of diseased and non-diseased patients. How can I analyse
>> the diagnostic properties of the score at different cut-off points
>> using appropriate method for the imputed sets of data?
>>
>> Roland Andersson
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/