Re: st: RE: Mean test in a Likert Scale

[Nick Cox <njcoxstata@gmail.com>]

I need a way of signalling to the researcher on the next questionnaire
I fill in:

I am trying to take your silly Likert scale as seriously as I can. If
you intend merely to reduce me to one of a dichotomy, I object. Recode
me as "missing on principle". If you are a Stata user, that can be
".p".

Nick

On Fri, Aug 31, 2012 at 10:01 AM, Seed, Paul <paul.seed@kcl.ac.uk> wrote:
> Dear Statalist,
>
> I sympathise with everything Nick has said.
>
> But there is a further point to be considered here:
> It is sometimes argued on psychometric grounds that
> when people answer a question on a Likert scale, there are two
> processes - firstly the answer (Yes/No)
> secondly a personal preference (or avoidance) of extreme views
> (what is sometimes called "response style").
>
> So mild-mannered Clark Kent may always give answers as 2 or 4,
> while Superman, the man of steel prefers 1 or 5.  But their
> meaning is the same. (Or compare mild-mannered Chinese students
> with more forthright Americans).
>
> If so, and assuming you want to catch the answer & drop the personal preference;
> it makes sense to collapse the Likert scale to 2 or 3 points (possibly treating
> 3 as separate from 4 and 5).
> But that depends on the question "Do people really behave like this?"
>
> Googling "collapse Likert scale response style" turned up some relevant references, which I have not read.
>
>
> Leonor Saravia <lmisaravia@gmail.com> wrote:
>
> Dear Nick and David,
>
> I really appreciate your reply, thank you.
>
> I read carefully your answers to my questions and as Nick says, my
> first question pointed to the fact that there could be the sense in
> which computing the mean score of a Likert scale is allowed. I have
> seen practical studies were the mean of this kind of scales are
> calculated and interpreted. However, there is also literature that
> indicates that, as the Likert scales are an ordinal-level measure, you
> should not calculate the mean of it. So, I am confused because I do
> not understand whether calculating and interpreting the mean of a
> Likert scale is correct or not.
>
> The data I have is disaggregated by individual (20000 observations) of
> a treatment and a control group, and has the answer for each of the 26
> questions, a number between 1 and 5, which are the values of a 5 point
> Likert scale from Disagree (1) to Agree (5).
>
> For instance, the first question (Q1) is: "Chilean people find
> entrepreneurial activities socially valuable" and the possible answers
> are:
>
> 1 - Strongly disagree
> 2 - Disagree
> 3 - Nor agree nor disagree
> 4 - Agree
> 5 - Strongly agree
>
> So, the database has this structure:
>
> Observation     Group            Q1  Q2  .....  Q25  Q26
> 1                     Treatment      1     5   ......   3      1
> 2                     Control           3    1   .......  2      5
> .
> .
> 19999             Control          5     2  ........ 4       3
> 20000             Treatment      3     2  ......... 5      4
>
>
> From this, one could calculate the mean of Q1 for the treatment and
> control group, but I do not know if the number obtained can be
> interpreted and even more, if one can test mean differences between
> both groups.
>
> Thank you very much for your help and advice.
>
> Best regards,
>
> Leonor
>
>>I mostly disagree with David here. In particular, his proposal to
>>collapse the Likert scales just throws away information in an
>>arbitrary manner.
>
>>I don't think his advice is even consistent. If it's OK to treat means
>>of Bernoulli distributions as valid arguments for a t test, why is not
>>OK to treat means of Likert scales as if they were?
>
>>It's true that the reference case for a t test is two paired normal
>>distributions, and Likert scales cannot be normal if only because
>>they are _not_ continuous, but there is always a judgment call on
>>whether summaries of the data will in practice work similarly.
>
>>A fair question is what exactly kind of advice is Leonor seeking? The
>>question presumably isn't really whether it is possible -- clearly it
>>is possible -- but perhaps somewhere between "Is it correct?" and "Is
>>it a good idea?"
>
>>Leonor's question appears to have the flavour of "I gather that this
>>is wrong. but is there a sense in which this is allowed?" The long
>>answer has to be that Leonor should tell us much more about the data
>>and the problem in hand if a good answer is to be given. If means make
>>sense as summary statistics, then comparing means with a t test is
>>likely to work well, but watch out.
>
>>David is clearly right in alluding to a purist literature in which you
>>are told as a matter of doctrine that ordinal data shouldn't be
>>summarised by means and so mean-based tests are also invalid. When
>>acting as academics, the same people work with grade-point averages
>>just like anybody else, at least in my experience.
>
>>There is also a pragmatist literature which points out that despite
>>all that, the sinful practice usually works well. Compare the t-test
>>with e.g. a Mann-Whitney-Wilcoxon test and it's very likely that the
>>P-values and z- or t-statistics will point to the same substantive
>>conclusion and indicate just about the same quantitative effect. It's
>>also likely that doing both tests will be needed because some reviewer
>>has been indoctrinated against t-tests here, and especially if anyone
>>is working with a rigid threshold (e.g. a 5% significance level).
>
>>Also, the behaviour of t-tests in cases like this can always be
>>examined by simulation, so no-one need be limited by textbook dogma
>>(or wickedness).
>
>>Nick
>
> On Fri, Aug 31, 2012 at 12:13 AM, David Radwin <dradwin@mprinc.com> wrote:
>> Leonor,
>>
>>No, you can't correctly calculate the mean of an ordinal-level measure
>>like the Likert scale you describe, although plenty of people do it
>> anyway.
>>
>> But you can use -ttest- with these data if you first collapse each
>> variable to a dichotomous (dummy) variable, because the mean of a
>> dichotomous variable is identical to the proportion where the value is 1.
>> As a guess, you might set the highest two values to 1, the lowest two
>> values to 0, and the middle value to missing to calculate the proportion
>> agreeing or somewhat agreeing.
>>
>> David
>> --
>> David Radwin
>> Senior Research Associate
>> MPR Associates, Inc.
>> 2150 Shattuck Ave., Suite 800
>> Berkeley, CA 94704
>> Phone:             510-849-4942
>> Fax: 510-849-0794
>>
>> www.mprinc.com
>>
>>
>>> -----Original Message-----
>>> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
>>> statalist@hsphsun2.harvard.edu] On Behalf Of Leonor Saravia
>>> Sent: Thursday, August 30, 2012 3:23 PM
>>> To: statalist@hsphsun2.harvard.edu
>>> Subject: st: Mean test in a Likert Scale
>>>
>>> Hello,
>>>
>>> I'm working with a survey that presents 26 questions and each of them
>>> has as possible answer a 5 point Likert scale from Disagree (1) to
>>> Agree (5). This survey was applied for a treatment and a control
>>> group.
>>>
>>> As far as I know, it is possible to analyze the information given only
>>> by the proportions of each answer; for instance, 25% agrees, 50%
>>> disagree, or so.
>>>
>>> I have two questions that maybe one of you have had before:
>>>
>>> a) Is it possible to calculate the mean score of a sample (treatment
>>> or control group) - adding the individual answers - when one is
>>> working with a Likert scale?
>>>
>>> b) If it is possible to calculate a mean score of a sample when using
>>> a Likert scale, to compare the answers of the treatment versus the
>>> control group, is it well done if I use the 'ttest' command?
>
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/