Re: st: Collapse command

[japellerano@neo.tamu.edu]

Thanks Austin, that worked.

Jose
----- Original Message -----
From: "Austin Nichols" <austinnichols@gmail.com>
To: statalist@hsphsun2.harvard.edu
Sent: Friday, March 4, 2011 4:07:25 PM GMT -06:00 US/Canada Central
Subject: Re: st: Collapse command

Jose <japellerano@neo.tamu.edu>:
Try putting
exit
as the last line of the do file. I am guessing the eof comes after
list with no line break so that Stata does not execute the last line.

On Fri, Mar 4, 2011 at 4:23 PM,  <japellerano@neo.tamu.edu> wrote:
> Thanks Erick,
>
> The set mem 30 g was unnecesary, the dataset is ~ 4. I'm running this on linux on the dept's server. And yes, I can see the "end of do file" line, but is not listing the collapsed data set.
>
> Let me read your message with care.
>
> Jose
> ----- Original Message -----
> From: "Eric Booth" <ebooth@ppri.tamu.edu>
> To: "<statalist@hsphsun2.harvard.edu>" <statalist@hsphsun2.harvard.edu>
> Sent: Friday, March 4, 2011 3:12:50 PM GMT -06:00 US/Canada Central
> Subject: Re: st: Collapse command
>
> <>
>
> I am puzzled by what you mean when you say it just ends.  A few questions:
>
> Does Stata print the line "end of do file" line after your last line?   If there is no message and you really are working on with ~ 30 G of data on a machine with at least 30G of RAM, I wonder  if Stata is still trying to perform the collapse (which could take a while with that many obs).   If you don't have 30G of RAM and you opened a 30G dataset, then it could take a really long time (is Stata still trying to run the command (do you see the spinning wheel in the bottom right corner of the Results window (assuming, based on the filepath in your code, that you are using a Mac) , or can you enter more commands in the command window and they run?  ).
>  I suspect you know that it could take a really long time which is why you -set virtual on- (although you should realize what -set virtual on- is (not) doing:  http://www.stata.com/statalist/archive/2007-06/msg00875.html )
>
> Does the data properly -collapse- (that is, if you -browse- the data, are they collapsed) and you just don't get the -list- output you expected?
>
> Try reducing the size of your dataset (-drop- or -sample- some of your obs)  and then -collapse- it to see if what happens.
>
> Try running this example and see if it works and if so, what is different in this example from your actual dataset (besides the size)?
> *******
> clear
>
> inp str11(campus) year studentid y_red
> "001903001" 2001 1  99
> "001903001" 2002 1  90
> "001903023" 2001 101  88
> "001903023" 2001 100  55
> "001903002" 2001 100  199
> "001903002" 2002 100  159
> end
>
> destring campus, replace
> format campus %09.0f
> /* note:
> assuming you're using AEIS/PEIMS data, you can replace "format campus %20.0f"
> with the command above to preserve the leading zeroes in the campus ids,
> but I prefer to keep them as string variables
> */
>
> collapse (count) y_red_count=y_red, by(campus year)
>
> list
> ********
>
> - Eric
> __
> Eric A. Booth
> Public Policy Research Institute
> Texas A&M University
> ebooth@ppri.tamu.edu
> Office: +979.845.6754
>
> On Mar 4, 2011, at 2:33 PM, <japellerano@neo.tamu.edu>
>  wrote:
>
>> Hello, I have a huge data set with student level data.
>>
>> I have been trying to collapse the data set at the school level, first time I did it, it worked, but that only one time.
>>
>> Here's the code:
>>
>> set mem 30g
>>
>> set virtual on
>>
>> use /home/jpellerano/scores_2003-2010_cleaned.dta
>>
>> destring campus, replace
>>
>> format campus %20.0f
>>
>> collapse (count) y_red_count=y_red, by(campus year)
>>
>> list
>>
>>
>> After reching the collapse line STATA ends the do file.
>>
>> I'll appreciate any suggestion.
>>
>> Thanks,
>>
>> Jose A. Pellerano
>> Texas A&M University
>> Dpt of Economics

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/