Re: st: How to reference results from a big dataset within a program

[Joerg Luedicke <joerg.luedicke@gmail.com>]

What do you mean with "pre-simulated dataset"? Is it _one_ fabricated
dataset, or results from 400k replications of a Monte Carlo
simulation?

Either way, it seems a bit puzzling why you would even need to include
a fixed set of data in your program? What information exactly are you
pulling from this dataset? Depending on the specific purpose of
including these data in your program, maybe you could derive some
analytic solution?

Joerg


On Tue, Aug 27, 2013 at 5:25 PM, Chen,Minxing <MChen3@mdanderson.org> wrote:
> Dear all,
>
> I submitted a manuscript and program to Stat Journal several months ago, and now got feedback from reviewer. I need a big help from your all regarding how to reference a big dataset within a program.
>
> Basically, in the program I submitted, I had to reference results from a big pre-simulated dataset (four variables, but around 400,000 observations). In my previous submission, I simply submitted the pre-simulated dataset with my program, and within the program I called up that simulated dataset by using code such as " use c:\ado\personal\simudata". I was hoping when people download the program from SSC, the pre-simulated dataset will be also downloaded to the directory "c:\ado\personal\".
>
> Now my reviewer indicated that I can't expect users to do that, I can't even tell the user to place the file there because such a directory may not be creatable for the user (e.g. they might not have a C: drive). The reviewer suggested me to find some other way to get the information in my pre-simulated dataset, such as incorporating the data into the program.
>
> I tried to copy of the simulated data within my program by using syntax such as "input x y z k", however, since there are so many observations (a little more than 400,000), and there are system limit for the maximum lines of syntax within a program (around 3500), I was not able to do this way. The reviewer also mentioned that I may use "Mata library" function, but I am pretty new to Stata Mata. Is there anyone that may be able to help regarding this issue?
>
> Thank you so much!
> Minxing Chen
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/