RE: st: Quantum mechanics in Stata

[Nick Cox <n.j.cox@durham.ac.uk>]

Stas: Thanks for a neat example. I hadn't focused at all on looking at individual reported percents, but it's a natural special case. 

I doubt that any cricket example I might devise would be more widely comprehensible. 

Nick 
n.j.cox@durham.ac.uk 

Stas Kolenikov

Nick,

I don't have a statistical quality control answer, but I have an
answer to the question you did not ask, namely: What is a better
season batting average, 0.338 or 0.833? Baseball experts would argue
the former is a better one, since the number of 0.833 is just an
unrealistic one, but may be a legitimate one if a player was at bat
only 6 times in a season, managing to hit 5 times, which has a
non-negligible probability of

. display comb(5,4)*0.2^4*0.8
.0064

for a player with a mediocre 20% long run batting average. The 0.338
number, though, can only appear if a player shows up in many games of
the season and is at bat at least

mata : quantum( 0.338, 0.0005 )
  65

times. And anybody who knows a bit about baseball (I don't) would know
that the long term batting averages above 0.3 would easily get you a
contract with any Major League team.

Feel free to substitute this with an equally obscure cricket example :)

On Mon, Nov 14, 2011 at 3:05 PM, Nick Cox <n.j.cox@durham.ac.uk> wrote:

> The heading may have led some here who expect something different -- or removed others who deleted something clearly off their own familiar territory. However, the problem is classical and real, the mechanics of how to estimate how many quanta, meaning number in a sample, there are in some data when that number is suppressed.
>
> There is a question at the end.
>
> Sometimes you are presented with a table of proportions or percents, but the total sample size has been suppressed. Naturally, we all know this is poor practice, at best a lack of thought and at worst an attempt to deceive. The reason I just got interested in this will itself be suppressed to protect the guilty, but I suspect no more than the first.
>
> I recollected a discussion in Becker, R.A., Chambers, J.M., Wilks, A.R. 1988. The new S language. Pacific Grove, CA: Wadsworth & Brooks/Cole on pp.272ff. They report a survey from a magazine in which percents favouring five vendors were 14.6, 12.2, 12.2, 7.3, 7.3. Here the repeated ties and the occurrence of 14.6 as twice 7.3 are consistent with a small sample size. So, what is the smallest sample size consistent with such percents? The main idea is just that rounding to a certain resolution (here 0.1) means a maximum error of half that, so we look for the smallest size that would fit with such rounding. (If percents have themselves been calculated or copied incorrectly or otherwise massaged, then clearly the problem becomes much more difficult.)
>
> There is S code on p.273 which can be translated into Mata. Becker and friends write code for proportions.
>
> /// 1.0.0 NJC 14 November 2011
> real quantum(real vector y, real scalar eps) {
>        real scalar n
>        real vector work, i
>        work = select(y, !missing(y))
>        n = 1
>
>        while (1) {
>                i = round(n :* work)
>                if (all((((work :- eps) :* n) :<= i) :&
>                        (((work :+ eps) :* n) :>= i))) {
>                        return(n)
>                        break
>                }
>                n++
>        }
> }
>
> end
>
> Their example with this code (and their S code too) give 41 as an answer.
>
> . mata :
> : pc = (.146,.122,.073)
>
> : quantum(pc, 0.0005)
>  41
>
> : percent = (14.6,12.2,7.3)
>
> : quantum(percent/100, 0.0005)
>  41
>
> : quantum(percent/100, 0.05/100)
>  41
>
> Clearly we should test it out on other cases too.
>
> . sysuse auto
> (1978 Automobile Data)
>
> . tab rep78
>
>     Repair |
> Record 1978 |      Freq.     Percent        Cum.
> ------------+-----------------------------------
>          1 |          2        2.90        2.90
>          2 |          8       11.59       14.49
>          3 |         30       43.48       57.97
>          4 |         18       26.09       84.06
>          5 |         11       15.94      100.00
> ------------+-----------------------------------
>      Total |         69      100.00
>
> . mata : quantum((2.9, 11.59, 43.48, 26.09, 15.94)/100, 0.005/100)
>  69
>
> So, this may be amusing, or even useful. (As said, you get the smallest consistent sample size: all integer multiples of that are also consistent with a given percent breakdown.)
>
> It seems too simple not to be much more widely known than it appears to be. Concerns about data quality and research abuses cross disciplines, so there should be a corresponding scattered literature.
>
> I know of an earlier discussion in Wallis, W.A. and Roberts, H.V. 1956. Statistics: a new approach. Glencoe, IL: Free Press.
>
> I also know of the rather different paper
>
> Kendall, D.G. 1974. Hunting quanta. Philosophical Transactions of the Royal Society of London. Series A 276: 231-266.
>
> What am I missing? (Cameron?)
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/