Re: st: RE: Mean test in a Likert Scale

[Nick Cox <njcoxstata@gmail.com>]

However, it is possible to broaden the debate in various ways.

First, you will probably be aware that there is a massive literature
on the relationship (or otherwise) between measurement scales and
statistics. My own favourites are as below, partly because I think
both are relatively clear and partly because you can trace dissenting
discussions:

Velleman, P.F. and Wilkinson, L. 1993.  Nominal, ordinal, interval,
and ratio typologies are misleading. The American Statistician 47:
65-72. See also follow-up under "Letters to the Editor"  in 47(4) and
48(1).

[I note incidentally that the comment by David J. Hand is reported as
if from "David J. Hand and Milton Keynes". That was a mistake: "Milton
Keynes" is not a person, but a place, the town where Hand's then
institution, the Open University, is based.]

Hand, D.J. 1996. Statistics and the theory of measurement.
Journal of the Royal Statistical Society. Series A (Statistics in
Society) 159: 445-492.

[In the discussion I appear side-by-side with my namesake D.R. Cox
(the great Sir David, no relative) saying very similar things.]

Second, at the level of descriptive statistics the usual argument from
purists is that means are not justified for Likert scales but that
medians are. That still leaves scope for interpolating medians and so
using more of the information in the data. See the thread started by
J. Taggert Brooks at

http://www.stata.com/statalist/archive/2009-01/msg00652.html

and -iquantile- from SSC. The link in the help file to that thread is
broken and will be fixed by correcting the help file.


On Fri, Aug 31, 2012 at 8:46 AM, Nick Cox <njcoxstata@gmail.com> wrote:
> Thanks for the extra detail, but I don't think it lets me add much to
> my own advice, except that with a sample size of 20000 very small
> differences are likely to seem significant by any particular
> significance test.
>
> Nick
>
> On Fri, Aug 31, 2012 at 5:08 AM, Leonor Saravia <lmisaravia@gmail.com> wrote:
>> Dear Nick and David,
>>
>> I really appreciate your reply, thank you.
>>
>> I read carefully your answers to my questions and as Nick says, my
>> first question pointed to the fact that there could be the sence in
>> which computing the mean score of a Likert scale is allowed. I have
>> seen practical studies were the mean of this kind of scales are
>> calculated and interpreted. However, there is also literature that
>> indicates that, as the Likert scales are an ordinal-level measure, you
>> should not calculate the mean of it. So, I am confused because I do
>> not understand whether calculating and interpreting the mean of a
>> Likert scale is correct or not.
>>
>> The data I have is desagregated by individual (20000 observations) of
>> a treatment and a control group, and has the answer for each of the 26
>> questions, a number between 1 and 5, which are the values of a 5 point
>> Likert scale from Disagree (1) to Agree (5).
>>
>> For instance, the first question (Q1) is: "Chilean people find
>> entrepreneurial activities socialy valuable" and the possible answers
>> are:
>>
>> 1 - Strongly disagree
>> 2 - Disagree
>> 3 - Nor agree nor disagree
>> 4 - Agree
>> 5 - Strongly agree
>>
>> So, the database has this structure:
>>
>> Observation     Group            Q1  Q2  .....  Q25  Q26
>> 1                     Treatment      1     5   ......   3      1
>> 2                     Control           3    1   .......  2      5
>> .
>> .
>> 19999             Control          5     2  ........ 4       3
>> 20000             Treatment      3     2  ......... 5      4
>>
>>
>> From this, one could calculate the mean of Q1 for the treatment and
>> control group, but I do not know if the number obtained can be
>> interpreted and even more, if one can test mean differences between
>> both groups.
>>
>> Thank you very much for your help and advice.
>>
>> Best regards,
>>
>> Leonor
>>
>>>I mostly disagree with David here. In particular, his proposal to
>>>collapse the Likert scales just throws away information in an
>>>arbitrary manner.
>>
>>>I don't think his advice is even consistent. If it's OK to treat means
>>>of Bernoulli distributions as valid arguments for a t test, why is not
>>>OK to treat means of Likert scales as if they were?
>>
>>>It's true that the reference case for a t test is two paired normal
>>>distributions, and Likert scales can not be normal if only because
>>>they are _not_ continuous, but there is always a judgment call on
>>>whether summaries of the data will in practice work similarly.
>>
>>>A fair question is what exactly kind of advice is Leonor seeking? The
>>>question presumably isn't really whether it is possible -- clearly it
>>>is possible -- but perhaps somewhere between "Is it correct?" and "Is
>>>it a good idea?"
>>
>>>Leonor's question appears to have the flavour of "I gather that this
>>>is wrong. but is there a sense in which this is allowed?" The long
>>>answer has to be that Leonor should tell us much more about the data
>>>and the problem in hand if a good answer is to be given. If means make
>>>sense as summary statistics, then comparing means with a t test is
>>>likely to work well, but watch out.
>>
>>>David is clearly right in alluding to a purist literature in which you
>>>are told as a matter of doctrine that ordinal data shouldn't be
>>>summarised by means and so mean-based tests are also invalid. When
>>>acting as academics, the same people work with grade-point averages
>>>just like anybody else, at least in my experience.
>>
>>>There is also a pragmatist literature which points out that despite
>>>all that, the sinful practice usually works well. Compare the t-test
>>>with e.g. a Mann-Whitney-Wilcoxon test and it's very likely that the
>>>P-values and z- or t-statistics will point to the same substantive
>>>conclusion and indicate just about the same quantitative effect. It's
>>>also likely that doing both tests will be needed because some reviewer
>>>has been indoctrinated against t-tests here, and especially if anyone
>>>is working with a rigid threshold (e.g. a 5% significance level).
>>
>>>Also, the behaviour of t-tests in cases like this can always be
>>>examined by simulation, so no-one need be limited by textbook dogma
>>>(or wickedness).
>>
>>>Nick
>>
>> On Fri, Aug 31, 2012 at 12:13 AM, David Radwin <dradwin@mprinc.com> wrote:
>>> Leonor,
>>>
>>>No, you can't correctly calculate the mean of an ordinal-level measure
>>>like the Likert scale you describe, although plenty of people do it
>>> anyway.
>>>
>>> But you can use -ttest- with these data if you first collapse each
>>> variable to a dichotomous (dummy) variable, because the mean of a
>>> dichotomous variable is identical to the proportion where the value is 1.
>>> As a guess, you might set the highest two values to 1, the lowest two
>>> values to 0, and the middle value to missing to calculate the proportion
>>> agreeing or somewhat agreeing.
>>>
>>> David
>>> --
>>> David Radwin
>>> Senior Research Associate
>>> MPR Associates, Inc.
>>> 2150 Shattuck Ave., Suite 800
>>> Berkeley, CA 94704
>>> Phone:             510-849-4942
>>> Fax: 510-849-0794
>>>
>>> www.mprinc.com
>>>
>>>
>>>> -----Original Message-----
>>>> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
>>>> statalist@hsphsun2.harvard.edu] On Behalf Of Leonor Saravia
>>>> Sent: Thursday, August 30, 2012 3:23 PM
>>>> To: statalist@hsphsun2.harvard.edu
>>>> Subject: st: Mean test in a Likert Scale
>>>>
>>>> Hello,
>>>>
>>>> I'm working with a survey that presents 26 questions and each of them
>>>> has as possible answer a 5 point Likert scale from Desagree (1) to
>>>> Agree (5). This survey was applyed for a treatment and a control
>>>> group.
>>>>
>>>> As far as I know, it is possible to analyze the information given only
>>>> by the proportions of each answer; for instance, 25% agrees, 50%
>>>> desagree, or so.
>>>>
>>>> I have two questions that maybe one of you have had before:
>>>>
>>>> a) Is it possible to calculate the mean score of a sample (treatment
>>>> or control group) - adding the individual answers - when one is
>>>> working with a Likert scale?
>>>>
>>>> b) If it is possible to calculate a mean score of a sample when using
>>>> a Likert scale, to compare the answers of the treatment versus the
>>>> control group, is it well done if I use the 'ttest' command?
>>
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/