Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: -project- and big data files slowing down -build-


From   Roberto Ferrer <[email protected]>
To   Stata Help <[email protected]>
Subject   st: -project- and big data files slowing down -build-
Date   Wed, 24 Jul 2013 01:57:20 +0100

User-written package -project- by Robert Picard and installed using:
net from http://robertpicard.com/stata

What is the recommended course of action if I have a big data file
that seems to slow down the project (re)build ?

According to the help file:

"The do(do_filename) build directive will not run do_filename if the
do-file has not changed and all files linked to it have not changed
since the last build."

So I imagine there's a -checksum- slowing down the build even if no
files change. I'm thinking of some option that would tell the build
process to ignore this specific file. This file is the first input in
the whole sequence (an -original-) and I'm sure it cannot change since
it is in a write-protected directory.

I suppose I can take this step out of the build and modify the
corresponding files. At the end of the project, I could stick it back
in. But a build directive like

        project, original(dta_filename) ignore_chsum

would be nice.

The data file is 1.4GB in size and a build with no changes is taking
around 30 seconds. I did an isolated -checksum- on the file and it's
over 24 seconds. Other than that one I have few (38 linked) and small
(<2mb) files.

Thanks,
Roberto
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index