RE: st: RE: Mean test in a Likert Scale

["David Radwin" <dradwin@mprinc.com>]

I agree with Nick that "don't use means for ordinal data" is a purist
stance, even if it is more honored in the breach than the observance.

I also agree that collapsing Likert scales as described below is throwing
away information. But I don't think it is arbitrary to consider "agree"
and "strongly agree" to be one category and "disagree" and "strongly
disagree" to be the other category.

There is a separate question of why one would measure agreement on a 5
point (or 7 point or 101 point or whatever) scale only to later collapse
it to yes/no, but analysts don't always have the ability to design the
survey themselves.

David
--
David Radwin
Senior Research Associate
MPR Associates, Inc.
2150 Shattuck Ave., Suite 800
Berkeley, CA 94704
Phone: 510-849-4942
Fax: 510-849-0794

www.mprinc.com


> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> statalist@hsphsun2.harvard.edu] On Behalf Of Nick Cox
> Sent: Friday, August 31, 2012 12:46 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: RE: Mean test in a Likert Scale
> 
> Thanks for the extra detail, but I don't think it lets me add much to
> my own advice, except that with a sample size of 20000 very small
> differences are likely to seem significant by any particular
> significance test.
> 
> Nick
> 
> On Fri, Aug 31, 2012 at 5:08 AM, Leonor Saravia <lmisaravia@gmail.com>
> wrote:
> > Dear Nick and David,
> >
> > I really appreciate your reply, thank you.
> >
> > I read carefully your answers to my questions and as Nick says, my
> > first question pointed to the fact that there could be the sence in
> > which computing the mean score of a Likert scale is allowed. I have
> > seen practical studies were the mean of this kind of scales are
> > calculated and interpreted. However, there is also literature that
> > indicates that, as the Likert scales are an ordinal-level measure, you
> > should not calculate the mean of it. So, I am confused because I do
> > not understand whether calculating and interpreting the mean of a
> > Likert scale is correct or not.
> >
> > The data I have is desagregated by individual (20000 observations) of
> > a treatment and a control group, and has the answer for each of the 26
> > questions, a number between 1 and 5, which are the values of a 5 point
> > Likert scale from Disagree (1) to Agree (5).
> >
> > For instance, the first question (Q1) is: "Chilean people find
> > entrepreneurial activities socialy valuable" and the possible answers
> > are:
> >
> > 1 - Strongly disagree
> > 2 - Disagree
> > 3 - Nor agree nor disagree
> > 4 - Agree
> > 5 - Strongly agree
> >
> > So, the database has this structure:
> >
> > Observation     Group            Q1  Q2  .....  Q25  Q26
> > 1                     Treatment      1     5   ......   3      1
> > 2                     Control           3    1   .......  2      5
> > .
> > .
> > 19999             Control          5     2  ........ 4       3
> > 20000             Treatment      3     2  ......... 5      4
> >
> >
> > From this, one could calculate the mean of Q1 for the treatment and
> > control group, but I do not know if the number obtained can be
> > interpreted and even more, if one can test mean differences between
> > both groups.
> >
> > Thank you very much for your help and advice.
> >
> > Best regards,
> >
> > Leonor
> >
> >>I mostly disagree with David here. In particular, his proposal to
> >>collapse the Likert scales just throws away information in an
> >>arbitrary manner.
> >
> >>I don't think his advice is even consistent. If it's OK to treat means
> >>of Bernoulli distributions as valid arguments for a t test, why is not
> >>OK to treat means of Likert scales as if they were?
> >
> >>It's true that the reference case for a t test is two paired normal
> >>distributions, and Likert scales can not be normal if only because
> >>they are _not_ continuous, but there is always a judgment call on
> >>whether summaries of the data will in practice work similarly.
> >
> >>A fair question is what exactly kind of advice is Leonor seeking? The
> >>question presumably isn't really whether it is possible -- clearly it
> >>is possible -- but perhaps somewhere between "Is it correct?" and "Is
> >>it a good idea?"
> >
> >>Leonor's question appears to have the flavour of "I gather that this
> >>is wrong. but is there a sense in which this is allowed?" The long
> >>answer has to be that Leonor should tell us much more about the data
> >>and the problem in hand if a good answer is to be given. If means make
> >>sense as summary statistics, then comparing means with a t test is
> >>likely to work well, but watch out.
> >
> >>David is clearly right in alluding to a purist literature in which you
> >>are told as a matter of doctrine that ordinal data shouldn't be
> >>summarised by means and so mean-based tests are also invalid. When
> >>acting as academics, the same people work with grade-point averages
> >>just like anybody else, at least in my experience.
> >
> >>There is also a pragmatist literature which points out that despite
> >>all that, the sinful practice usually works well. Compare the t-test
> >>with e.g. a Mann-Whitney-Wilcoxon test and it's very likely that the
> >>P-values and z- or t-statistics will point to the same substantive
> >>conclusion and indicate just about the same quantitative effect. It's
> >>also likely that doing both tests will be needed because some reviewer
> >>has been indoctrinated against t-tests here, and especially if anyone
> >>is working with a rigid threshold (e.g. a 5% significance level).
> >
> >>Also, the behaviour of t-tests in cases like this can always be
> >>examined by simulation, so no-one need be limited by textbook dogma
> >>(or wickedness).
> >
> >>Nick
> >
> > On Fri, Aug 31, 2012 at 12:13 AM, David Radwin <dradwin@mprinc.com>
> wrote:
> >> Leonor,
> >>
> >>No, you can't correctly calculate the mean of an ordinal-level measure
> >>like the Likert scale you describe, although plenty of people do it
> >> anyway.
> >>
> >> But you can use -ttest- with these data if you first collapse each
> >> variable to a dichotomous (dummy) variable, because the mean of a
> >> dichotomous variable is identical to the proportion where the value
is
> 1.
> >> As a guess, you might set the highest two values to 1, the lowest two
> >> values to 0, and the middle value to missing to calculate the
> proportion
> >> agreeing or somewhat agreeing.
> >>
> >> David
> >> --
> >> David Radwin
> >> Senior Research Associate
> >> MPR Associates, Inc.
> >> 2150 Shattuck Ave., Suite 800
> >> Berkeley, CA 94704
> >> Phone:             510-849-4942
> >> Fax: 510-849-0794
> >>
> >> www.mprinc.com
> >>
> >>
> >>> -----Original Message-----
> >>> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> >>> statalist@hsphsun2.harvard.edu] On Behalf Of Leonor Saravia
> >>> Sent: Thursday, August 30, 2012 3:23 PM
> >>> To: statalist@hsphsun2.harvard.edu
> >>> Subject: st: Mean test in a Likert Scale
> >>>
> >>> Hello,
> >>>
> >>> I'm working with a survey that presents 26 questions and each of
them
> >>> has as possible answer a 5 point Likert scale from Desagree (1) to
> >>> Agree (5). This survey was applyed for a treatment and a control
> >>> group.
> >>>
> >>> As far as I know, it is possible to analyze the information given
only
> >>> by the proportions of each answer; for instance, 25% agrees, 50%
> >>> desagree, or so.
> >>>
> >>> I have two questions that maybe one of you have had before:
> >>>
> >>> a) Is it possible to calculate the mean score of a sample (treatment
> >>> or control group) - adding the individual answers - when one is
> >>> working with a Likert scale?
> >>>
> >>> b) If it is possible to calculate a mean score of a sample when
using
> >>> a Likert scale, to compare the answers of the treatment versus the
> >>> control group, is it well done if I use the 'ttest' command?

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/