Re: st: Comparing risk scores

[Richard Goldstein <richgold@ix.netcom.com>]

Karin,

I suggest you might want to read Sullivan, LM, et al. (2004),
"Presentation of multivariate data for clinical use: the Framingham
study risk score functions," _Statistics in Medicine_, 23: 1631-1660,
which describes how the Framingham people came up with their risk scores

Rich

On 10/18/11 8:18 AM, K Jensen wrote:
> Hi Nick
> 
> Thanks for your reply.  It's actually a bit more complicated than
> that.  We are trying to construct a "best" single score that would be
> simple and used clinically.  The elements that are summed to make the
> score (0,1,2,3 etc) are derived from various clinical measurements.
> They are dichotomised by choosing the cutpoint that maximises the sum
> of sensitivity+specificity.  Only those binary variables significant
> in a univariate logistic regression are proposed for the model.
> 
> I am wanting to choose the "best" model, that is useful for
> clinicians.  If we had 7 binary variables, say, I would look at all
> possibilities of choosing different combinations of the sums of them.
> E.g. 1, 2, 3, 4, 5, 6, 7,1+2,1+3,1+4,1+5,1+6,1+7, 2+3, 2+4,... up to
> 1+2+3+4+5+6+7.  I would like to use the optimal score based on this
> method, but don't know how to measure optimality.
> 
> Best wishes,
> 
> Karin
> 
> On 18 October 2011 12:36, Nick Cox <njcoxstata@gmail.com> wrote:
>> I would recast this as a -logit- or -logistic- problem in which your
>> outcome is dead or alive. Depending on how you think about your
>> scores, they define predictors to be treated as they come or
>> predictors to be treated as a set of indicator variables (or in some
>> cases both).
>>
>>  I don't think you are restricted to using one score or the other as predictor.
>>
>> Nick
>>
>> On Tue, Oct 18, 2011 at 12:11 PM, K Jensen <k.x.jensen@gmail.com> wrote:
>>> Maybe this is more of a stats question than a Stata one, but there are
>>> such a lot of good brains here...
>>>
>>> We are constructing point scores to indicate severity of risk  Death
>>> is the outcome. What is the best way of measuring the usefulness of
>>> the score?  The aim is to show a good gradient of risk.  Say the
>>> results for two different scores were:
>>>
>>> Score  Dead  Alive    %dead    Totals
>>> 0        12    136      9.9%      145
>>> 1        18    126     15.4%      144
>>> 2        18     62     26.2%       81
>>> 3        10      9     57.1%       20
>>> 4         2      0    100  %        3
>>> -------------------------------------
>>> Total:   60    333                393
>>>
>>> Score  Dead  Alive    %dead    Totals
>>> 0         8    174      4.6%      182
>>> 1        21    143     12.8%      164
>>> 2        22     19     53.7%       41
>>> 3         5      1     83.3%        6
>>> -------------------------------------
>>> TOTAL:   60    333                393
>>>
>>> Which is the better score?  What is the best way to measure its
>>> predictive power?  I understand that ROC type analysis doesn't really
>>> apply here.  Some measure of R-squared?  AIC?
>>>
>>> Thankyou
>>>
>>> Karin
>>>
>>> PS) I have made up the data, so the numbers don't quite add up.  It is
>>> meant to be two different, competing scores on the same people.
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/