Re: st: RE: Interval regression with skewed data

[Nick Cox <njcoxstata@gmail.com>]

So, your problem is essentially whether the variably censored
distribution of your outcome differs by region. Linearity is thus not
an issue as you would need to represent your categorical variable
region by a set of indicator variables.

Depending on how much censoring there is you might be able to work out
median and quartiles and produce partial box plots to get a feeling
for skewness.

Some people in ecology use log(count + 1) as a transform. People
outside that field tend to be sniffy about it.

Alan's suggestion remains a very good one in principle.

Nick

On Tue, Jan 10, 2012 at 8:40 AM,  <Gillian.Frost@hsl.gov.uk> wrote:
> Hello Nick, Alan,
>
> Thank you both for your replies.
>
> Nick, I apologise for not being clear in my original posting.  My
> outcome/dependent variable is the number of colony forming units per ml,
> and my predictor/independent variable is the region (North West, North
> East, South East England,...) within which the sample was taken.  I
> gravitated towards interval regression because I have some observations
> that are left censored and some that are right censored but the censoring
> value is not always the same, and I started to think about survival
> analysis because I had seen a suggestion where this could be used to
> perform interval regression when the Normality assumption was violated.
> Unfortunately, my outcome has some zero counts and so I cannot really use
> the logarithmic transform.  I am more than happy to consider other methods
> of analysis if you have any ideas?
>
> Alan, I am afraid that what you suggest is probably outside of my
> programming and statistical expertise, and would also take me longer than
> the time I have to look at this problem.
>
> Many thanks,
>
> Gillian
>
>
>
>
>
> From:   Nick Cox <n.j.cox@durham.ac.uk>
> To:     "'statalist@hsphsun2.harvard.edu'"
> <statalist@hsphsun2.harvard.edu>
> Date:   09/01/2012 16:15
> Subject:        st: RE: Interval regression with skewed data
> Sent by:        owner-statalist@hsphsun2.harvard.edu
>
>
>
> I'd be more worried about violating linearity of functional form than
> normality of errors, but you say nothing about that. Nor do you say
> anything about what your predictors are.
>
> I can't see from your discussion that it can be a choice between interval
> regression and some kind of survival analysis. What you have doesn't sound
> to me at all like a survival analysis problem.
>
> However, assuming the first, you could transform before you use -intreg-.
> Your limits just transform to limits on your transformed scale. From other
> experiences with hydrological data I would reach for a logarithmic
> transform as first port of call. You would need to back-transform
> afterwards.
>
> Nick
> n.j.cox@durham.ac.uk
>
> Gillian.Frost@hsl.gov.uk
>
> I am struggling with an analysis and would like your insight.  I think
> that I am looking at using interval regression but there are certain
> aspects of the data that are worrying me.  First some background...
>
> A number of water samples have been taken from around the UK, and a
> microbiological examination of the water has been undertaken.  Whenever a
> sample is sent to a lab, a whole suite of tests are done to count the
> number of colony forming units of various organisms.  I therefore have a
> number of outcomes, whose units are the number of colony forming units per
>
> ml.  The aim of this part of the analysis is to compare the organism
> levels found in different regions of the UK.
>
> Some observations are left censored (0-6% depending on the outcome) - ie
> <1 CFU/ml, or <10 CFU/ml - and some are right censored (0-59%) - ie. >3000
>
> CFU/ml.  The censoring point varies,and so I thought that I would have to
> use interval regression (Stata's -intreg-).
>
> However, the data are not Normally distributed (which is an assumption of
> interval regression), but are positively skewed with some outcomes having
> a high number of zero counts (one has 75% zeros!).  In the book by J S
> Long (Regression models for categorical and limited dependent variables,
> 2007), there was a discussion about how accelerated failure time (AFT)
> models can be used to perform interval regression when the data are not
> Normally distributed, but there was no example of how to do this.
> Unfortunately I no longer have the book to provide you with the page
> reference.
>
> I have found a user written command -intcens-, which can perform
> interval-censored survival analysis and fits a number of different
> distributions, but I cannot find any documentation or examples of its use
> (apart from the help file).
>
> Does anyone have any examples of using AFT models to perform interval
> regression or examples of using -intcens-?  Or do you think that there is
> a better way I could be handling the data?
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/