st: RE: -cloglog- memory & -stcurve- median : was -svy stocx- attained age - issues with posting

["Muhuri, Pradip (SAMHSA/CBHSQ)" <Pradip.Muhuri@samhsa.hhs.gov>]

Hello,

I am new to the Statalist. 

I still don't understand why my earlier messages except two of them did get posted to the Statalist.  Our IT service desk has confirmed that my all messages have left our domain successfully and that they reached the recipient domain (i.e., Statalist). 

Although those e-mails did not get posted to the Statlist, the recent two e-mails reached Steve's address.  My apologies to him for any inconvenience this may have caused.  However, I appreciated receiving his excellent response on the issues I have been dealing with.
  
Any help toward resolving the issue would be appreciated.

Thanks,
    

Pradip K. Muhuri, PhD
SAMHSA/CBHSQ
1 Choke Cherry Road, Room 2-1071
Rockville, MD 20857
Tel: 240-276-1070
Fax: 240-276-1260

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Steve Samuels
Sent: Tuesday, August 06, 2013 5:26 PM
To: statalist@hsphsun2.harvard.edu
Subject: st: -cloglog- memory & -stcurve- median : was -svy stocx- attained age 

Pradip:

Statalist is not a place for private conversations. Nobody will be able to follow this thread unless you are clear about what you are doing.

In my response to your private email, I asked you to send your original private post to Statalist. Please now describe the study, data, and study questions so that others might help you.

For specific issues, show code and results as the Statalist FAQ request, detail the models you tried, the relevant output, and give essential information, such as the N, number of deaths, descriptive stats on your primary exposure variable, and a 2 x 2 table cross-tabulating death and exposure.

To bring others up to date, the d* and dur* variables Pradip refers to are period variables for -cloglog-, as detailed in Lesson 6 of https://www.iser.essex.ac.uk/resources/survival-analysis-with-stata
The "spell" dataset, is the result of an -expand- operation which contains one observation for each period a person is at risk:

About memory problem, I have only generic advice:

. Upgrade your OS to one that can address more memory

. Add physical memory

As we know more details, perhaps other ideas may suggest themselves.

To estimate the median, compute it "by hand" for a single curve; then try to write code that will automate what you did. If you can't, "by hand" may be good enough. Just be aware that, because of the grouping, you'll need to linearly interpolate between quarter end points or to fit flexible parametric models. I pointed the way to creating expected values in an earlier post. For a way of adding points to a graph, see:
http://www.stata.com/statalist/archive/2008-02/msg01145.html

By the way: you are not correct in assuming that date of interview can be located only by year and quarter. The NHIS data sets contain an "assignment week" of the quarter, and instructions are to finish the survey by that week, or no later than a bit longer than 14 days. Thus you can easily locate interview date to the most likely month, that of the assignment week. I'm not sure that will you do any good unless you can work with the restricted data that has date of death.

See:
http://www.amstat.org/sections/srms/proceedings/papers/1992_048.pdf)
http://www.amstat.org/sections/srms/proceedings/y2001/Proceed/00040.pdf

You are welcome for the advice I've given so far, but to quote a recent post from Nick Cox:

"Better to think that you are replying to the list. Anyone who replies to you is not volunteering to provide dedicated support, just trying to push a discussion forward. Nor will they necessarily write lots of code for you...What you want requires custom programming..."

Regards,

Steve



On Aug 5, 2013, at 3:54 PM, Muhuri, Pradip (SAMHSA/CBHSQ) wrote:

Steve, Thank you so much for your continued support, advice, and excellent reference materials (Thanks to Professor Jenkins for making his materials available online). My apologies for e-mail transmission problems, source yet to be ascertained. Since my recent e-mail did also not get posted, I am resending it, with no copying or pasting this time.

I would like to run models with both -svy stcox- and -svy cloglog- for comparison purposes. But, I would like to report results from -cloglog- models.

Issues:

1) -cloglog-: The spell-quarter data file is created (compressed -
5,485,946 obs 70 vars including d1-d39 dummies and dur1-dur10 variables, with -set memory 725m - can't go beyond this limit). Failed to run models due to memory issue. Any advice?

2) -syv:stcox-: Plotted the survival curve [...at1() ... at2 ()] and then saved the results in a .dta file that provides me three variables (_t, surv2, and surv3). I am looking for the sample code to calculate the following:

	(a) the median survival time, that is, at at with (_t) half of the
people survived (based 	on surv2 and surv3); and

	(b) the arithmetic mean of the survival time (_t) or life expectancy.

Your advice toward resolving the issues will be highly appreciated.

Regards,

Pradip

Pradip K. Muhuri, PhD SAMHSA/CBHSQ 1 Choke Cherry Road, Room 2-1071 Rockville, MD 20857 Tel: 240-276-1070 Fax: 240-276-1260

-----Original Message----- From: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Steve Samuels
Sent: Wednesday, July 31, 2013 6:30 PM To:
statalist@hsphsun2.harvard.edu Subject: Re: st: -svy stocx- attained age

Pradip:

You are encountering a version of problem that I diagnosed earlier this month at http://www.stata.com/statalist/archive/2013-07/msg00644.html,
where I referred to
http://www.stata.com/support/faqs/statistics/stcox-producing-missing-
standard-errors/ "4) Covariate does not vary within death event risk sets."

In your case, people in the same age group at baseline will have the same attained age at all points of followup. You can use either age at baseline or time as attained age, but not both.

I'd recommend age at baseline, as you know this pretty precisely, a gain that doesn't transfer to attained age. But the use of only five age groups loses a lot of information; try fractional polynomial regression
(-fp-) on continuous age.

In a post to me, you stated that you have a maximum of 10 years of follow-up and, that for dates of death and interview, you have only year and quarter. Your solution is to assign the midpoint date of each quarter. This could work, but violates the assumption of -stcox- that times are essentially continous. The measurement error in follow-up time could be as much as ±3 months and will probably bias the estimated coefficients. Moreover, if someone died in < 3 months after baseline, you would be assigning start and death to the same date, and -stset- will drop the observation.

Therefore I suggest that you also do a grouped hazard analysis with
-cloglog- which accepts a -svy- prefix. (-stpm2-, as you reminded me, does not.). With -cloglog-, assign the person who died < 3 months after baseline to the first period. For more details, see the the Lesson 6 link to discrete data analysis on Stephen Jenkins's fine web page "Survival analysis with Stata"
(http://www.iser.essex.ac.uk/survival-analysis )

Steve

On Jul 31, 2013, at 12:53 PM, Muhuri, Pradip (SAMHSA/CBHSQ) wrote:

Hello,

I am new to the Statalist. In response to my first posted e-mail to the List, and I got a reply from Steve, with insightful comments and advice.
At this point, I need your help with two issues.

1) All my subsequent e-mails (content --copied from the Stata log file - in plain text, not html) have bounced back to me. I don't understand what I am doing wrong.

2) I am using -svy:stcox- models, with attained age as the time scale. I have successfully run several models. However, the addition of a 5-category factor (attained age) to the model gives me the following error message: "flat region resulting in a missing likelihood error occurred when svy executed stcox last estimates not found". Sorry I am not pasting the content from the log file, fearing that this e-mail will also bounce back.

Thanks,

Pradip

This not a private conversation between us. Nobody will be able to follow this thread unless you are clear about what you are doing. I asked you to send your original private post to Statalist. Please do so now. Include the study description and goals.

For specific issues, please show code and results as the Statalist FAQ request, detail the models you tried, and give essential information, such as the N, number of deaths, descriptive stats on your primary exposure variable, and a 2 x 2 table cross-tabulating death and exposure.

I already suggested that attained-age was

To bring others up to date, the d* and dur* variables Pradip refers to are period variables for -cloglog-, as detailed in Lesson 6 of https://www.iser.essex.ac.uk/resources/survival-analysis-with-stata

The "spell" dataset, is the result of an -expand- operation which contains one observation for each time a person is at risk:

As to your memory problem, I have only generic advice:

. Upgrade your OS to one that can address more memory

. Add physical memory

As we know more details, perhaps other ideas may suggest themselves.

Your other questions: To estimate the median, compute it "by hand" for a single curve; then try to write code that will automate what you did. If you can't, "by hand" may be good enough. Just be aware that, because of the grouping, you'll need to linearly interpolate to locate median to the nearest month. I pointed the way to creating expected values in an earlier post.

You are not correct in assuming that date of interview can be located only by year and quarter. The NHIS data sets contain an "assignment week" of the quarter, and instructions are to finish the survey by that week, or no later than a bit longer than 14 days. Thus you can easily locate interview date to the most likely month, that of the assignment week. I'm not sure that will you do any good.

See:
http://www.amstat.org/sections/srms/proceedings/papers/1992_048.pdf)
http://www.amstat.org/sections/srms/proceedings/y2001/Proceed/00040.pdf

You are welcome for the advice I've given, but to quote a recent post from Nick Cox:

"Better to think that you are replying to the list. Anyone who replies to you is not volunteering to provide dedicated support, just trying to push a discussion forward. Nor will they necessarily write lots of code for you. What you want requires custom programming..."



Regards,

Steve



On Aug 5, 2013, at 3:54 PM, Muhuri, Pradip (SAMHSA/CBHSQ) wrote:

Steve, Thank you so much for your continued support, advice, and excellent reference materials (Thanks to Professor Jenkins for making his materials available online). My apologies for e-mail transmission problems, source yet to be ascertained. Since my recent e-mail did also not get posted, I am resending it, with no copying or pasting this time.

I would like to run models with both -svy stcox- and -svy cloglog- for comparison purposes. But, I would like to report results from -cloglog- models.

Issues:

1) -cloglog-: The spell-quarter data file is created (compressed -
5,485,946 obs 70 vars including d1-d39 dummies and dur1-dur10 variables, with -set memory 725m - can't go beyond this limit). Failed to run models due to memory issue. Any advice?

2) -syv:stcox-: Plotted the survival curve [...at1() ... at2 ()] and then saved the results in a .dta file that provides me three variables (_t, surv2, and surv3). I am looking for the sample code to calculate the following:

	(a) the median survival time, that is, at at with (_t) half of the
people survived (based 	on surv2 and surv3); and

	(b) the arithmetic mean of the survival time (_t) or life expectancy.

Your advice toward resolving the issues will be highly appreciated.

Regards,

Pradip


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/