This topic of how to use and report baseline data came up on the allstat
list a couple of years ago. I reproduce below what I contributed to that
debate, principally because I criticized the Assman paper. Readers may
find it as, or more, convenient to go to the website and see the messages
in context: www.jiscmail.ac.uk, list allstat, archives for Jan/Feb 2001.
I believe the phrase I used, "medics who dabble in statistics" was in part
prompted by the case of Prof Roy Meadow. Readers outside the UK may
care to search on his name for background and updates to the various
prosecutions where his misuse of inference and statistics has caused
several mothers to be unfairly jailed.
RAR
----
Date: Mon, 5 Feb 2001 12:07:39 +0000
From: "R. Allan Reese" <[email protected]>
Subject: Query: Baseline characteristics, how to use
Sue Bogle summarised responses:
> So the answer is: don't do it.
I am confused, and alarmed that distorted advice might build into a myth.
The matter seems to arise from using "hypothesis testing" as a
shibboleth. I recently commented on a student's thesis chapter where,
indeed, baseline characteristics were examined one by one, tabulated
and tested. My comment was that these were not "results" as they were a
consequence of the design of the study, but nevertheless the tests were
appropriate. They should have been reported succinctly for the record,
something like: "the demographic characteristics of the two groups were
checked for comparability, and on standard tests did not differ: age
(t-test, p>0.1), sex (chi-square, p>0.9) ..."
The point is that in this situation the data are expected to be consistent
with the null hypothesis. Too many users assume that the only use of a
statistical test is to find "significant" results.
However, rather than rely upon a few statistical tests, I would advise
full examination of the baseline measures to check assumptions, and look
for patterns and exceptions. Some of the advice quoted in the summary
message smacked of mechanistic processing, suitable for routine quality
control but not for research investigations.
Am I mad, or what?
-----
Date: Mon, 5 Feb 2001 15:46:03 +0000
From: "R. Allan Reese" <[email protected]>
Subject: Baseline characteristics: a second conclusion
Josie Sandercock <[email protected]> recommended: Assman et al,
Lancet 2000 - Subgroup analysis and other (mis)uses of baseline
characteristics (25 March, pp1064-1069)
This paper does indeed clarify the discussion, and spells out that its
authors found in a survey of the medical literature numerous examples of
poor statistical practice. These included not just the use of baseline
data, but also subgroup analysis, the "exaggerated claims of treatment
effects arising from post-hoc emphases across multiple analyses" and "lack
of clarity" in explaining covariate-adjusted analyses especially when
interactions occurred.
Assman's recommendation on baseline comparisons coincides exactly with
mine: "Although reports should show in appropriate detail the types of
patient included, the baseline comparisons across treatments *need not be
so extensive* [as was being published].
However, Assman's comment on covariate adjustment shows "lack of clarity"
verging on delphic. "Adjustment for baseline factors with treatment
imbalances is unimportant, unless such factors relate to outcome." In
other words, it is *essential* to test covariates in the model. If you
thought they could not affect the outcome, why measure or record them? On
the other hand, when reporting results, *report the results*, with
suitable emphasis that relates to clinical importance, not abstract P
values.
I am pleased, therefore, to confirm my sanity, and to suggest that
medics who dabble in statistics are about as useful as statisticians who
offer ad hoc diagnoses of medical conditions.
In contrast to this morning's summary, and mirroring the advice offered in
the Peto reference (Lancet. 1999 Jul 3;354(9172):73. quoted by Paul Seed),
the received wisdom should be, *do the analysis* and *interpret the
results*. Then choose what and how to report. And that is not an
invitation to cherry pick whatever results suited your prejudice, but a
recognition that any report is produced by a process of selection and
ordering.
--------------------------------------------------------------
R Allan Reese Email: [email protected]
Graduate School
University of Hull
Tel +44 1482 466845 Fax: +44 1482 466436
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
