Re: st: Alternative to coefficient of variation (CV)/relative standard error as a measure of estimate's reliability/stability

[Nick Cox <njcoxstata@gmail.com>]

Exactly. I have a small collection of textbook references recommending
summary of temperatures in deg Celsius by a coefficient of variation.
Here not only does the CV explode as the mean approaches 0, it can be
negative as mean temperatures can naturally also be negative. The same
textbooks usually explain elsewhere that some calculations only make
sense for ratio scale measurements. Some of them recommend using deg
Fahrenheit as the "solution" to this problem, regardless of the fact
that elsewhere they cite Celsius and Fahrenheit scales as canonical
examples of interval scale measurement.

I've seen iqr/median used as an alternative to CV, but as far as you
are concerned it has the same problems.

Using a CV is essentially equivalent to an argument that differences
should be assessed on a logarithmic scale.

Nick

On Wed, Jun 13, 2012 at 6:42 AM, David Radwin <dradwin@mprinc.com> wrote:
> Nick,
>
> Thank you for your thoughtful contribution. I personally agree with your criticism of a one-size-fits-all solution, but that is the sort of statistic I am seeking nonetheless. I would very much appreciate suggestions, however imperfect they may be, from other Statalisters.
>
> Incidentally, an even more thorny situation is when the estimate is zero or close to zero, perhaps because it has been mean-deviated (demeaned) or is a measure of change when there is no change. Then the CV is undefined or essentially infinity!
>
> David
> --
> David Radwin
> Senior Research Associate
> MPR Associates, Inc.
> 2150 Shattuck Ave., Suite 800
> Berkeley, CA 94704
> Phone: 510-849-4942
> Fax: 510-849-0794
>
> www.mprinc.com
>
> ----- Original Message -----
> From: "Nick Cox" <njcoxstata@gmail.com>
> To: statalist@hsphsun2.harvard.edu
> Sent: Tuesday, June 12, 2012 5:57:44 PM
> Subject: Re: st: Alternative to coefficient of variation (CV)/relative standard error as a measure of estimate's reliability/stability
>
> A simple-minded but I think reasonable first stab at this issue is
> that the CV is a fair criterion if SD (specifically SE) is indeed
> proportional to mean (generally estimate), so that SD / mean is
> approximately constant in a given kind of situation.
>
> If that is not true as a kind of first approximation then information
> is needed on the relation between variability and estimate to make a
> better judgement.
>
> One example that I studied in excruciating detail several years ago
> was the variability of annual rainfall in which many people had been
> led from the  observation that variability of rainfall as measured by
> SD tends to increase with the mean to conclude that the CV was in some
> sense a better summary. However, in many cases it is more nearly true
> that SD varies as a power around 0.7 of the mean, in which case CV
> over-compensates. CV is, as you imply, even less appropriate for
> proportions.
>
> I'm not sure this will help you at all. I became sensitive to
> inappropriate uses of the CV years ago when I encountered examples
> even worse than those with annual rainfall and my best summary is that
> when the CV is not appropriate there is not usually a simpler or even
> different alternative that works well at all; rather analysis has to
> look directly at the structure of variability.
>
> Nick
>
> On Wed, Jun 13, 2012 at 12:37 AM, David Radwin <dradwin@mprinc.com> wrote:
>
>> I hope you will indulge me this statistics question that is not strictly
>> Stata-specific. I am looking for a statistic measuring an estimate's
>> reliability or stability as an alternative to the coefficient of variation
>> (CV), also known as the relative standard error. The CV is the standard
>> error of an estimate (proportion, mean, regression coefficient, etc.)
>> divided by the estimate itself, usually expressed as a percentage. For
>> example, if a survey finds 15% unemployment with a 6% standard error, the
>> CV is .06/.15 = .4 = 40%.
>>
>> Some US government agencies flag or suppress as unreliable any estimate
>> with a CV over a certain threshold such as 30% or 50%. But this standard
>> can be arbitrary (for example, 85% employment would have a much lower CV
>> of .06/.85 = 7%) and has other limitations.
>>
>> Can anyone suggest an alternative measure of stability or reliability?

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/