st: How to reference results from a big dataset within a program

["Chen,Minxing" <MChen3@mdanderson.org>]

Dear all,

I submitted a manuscript and program to Stat Journal several months ago, and now got feedback from reviewer. I need a big help from your all regarding how to reference a big dataset within a program.

Basically, in the program I submitted, I had to reference results from a big pre-simulated dataset (four variables, but around 400,000 observations). In my previous submission, I simply submitted the pre-simulated dataset with my program, and within the program I called up that simulated dataset by using code such as " use c:\ado\personal\simudata". I was hoping when people download the program from SSC, the pre-simulated dataset will be also downloaded to the directory "c:\ado\personal\". 

Now my reviewer indicated that I can't expect users to do that, I can't even tell the user to place the file there because such a directory may not be creatable for the user (e.g. they might not have a C: drive). The reviewer suggested me to find some other way to get the information in my pre-simulated dataset, such as incorporating the data into the program. 

I tried to copy of the simulated data within my program by using syntax such as "input x y z k", however, since there are so many observations (a little more than 400,000), and there are system limit for the maximum lines of syntax within a program (around 3500), I was not able to do this way. The reviewer also mentioned that I may use "Mata library" function, but I am pretty new to Stata Mata. Is there anyone that may be able to help regarding this issue?

Thank you so much!
Minxing Chen

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/