Re: st: compiling mata from ado file

["William Gould, StataCorp LP" <wgould@stata.com>]

Laszlo <sandorl@gmail.com> asks abouit compiling Mata code from an 
ado-file.  He tries a structure like this, 

        ---------------------------------- mycmd.ado ---
        program mycmd
                ...
                mata: myfunction()
                ...
        end

        mata: 
        void myfunction(...) 
        {
                ...
        }
        end
        mata mosave myfunction(), replace
        ---------------------------------- mycmd.ado ---


Laszlo comments, 

> My code runs fine if I precompile the Mata code and the ado only looks
> for the .mo files. But I don't want to do that as I don't want version
> control on the Mata code (newer versions will compile faster code,
> no?).

Laszlo approach will not work.  Laszlo has to make a choice, 

      1.  Precompile the Mata code. 

      2.  Include the Mata code in the ado-file but then do not 
          attempt to save the function for external use.


1.  Precompile the Mata code
----------------------------

The ado-file then looks like this:
     
        ---------------------------------- mycmd.ado ---
        program mycmd
                version 12
                ...
                mata: myfunction()
                ...
        end
        ---------------------------------- mycmd.ado ---

When -mycmd- executes and it comes time to run the line 
-mata: myfunction()-, Stata will look for -myfunction()- and 
find it in a l<name>.mlib library or a myfunction.mo file.
If the function is not found, Stata will complain. 

Aside:  I added -version 12- as the first executable line 
of the ado-file.  I did that so that mycmd.ado will work 
with future versions of Stata. 


2.  Include Mata code in the ado-file
-------------------------------------

The ado-file then looks like this:

        ---------------------------------- mycmd.ado ---
        program mycmd
                version 12
                ...
                mata: myfunction()
                ...
        end

        version 12
        mata: 
        void myfunction(...) 
        {
                ...
        }
        end
        ---------------------------------- mycmd.ado ---


In the above example, -myfunction()- is private to mycmd.ado.  If
there is some other -myfunction()- in another ado-file, or outside of
ado-files, either aloready loaded into Mata, or in a l<name>.mlib
library, or a myfunction.mo file, that will be ignored.  When -mycmd-
calls -myfunction()-, it is the private -myfunction()- that will run
in all cases.

I recommend this approach.  

The first time you run -mycmd-, Stata loads mycmd.ado.  Stata stores 
away -mycmd- for later execution, and then compiles myfunction(), 
and stores that away with -mycmd-.   Then Stata executes -mycmd-. 

The second time you run -mycmd-, Stata finds -mycmd- in memory, 
accesses it, and then executes -mycmd-, skipping the compile step. 

Or, the second time, or the third, or ...,  Stata doesn't find -mycmd-
in memory because memory got tight and Stata discarded the compiled 
and ready-to-run mycmd/myfunction() combination.  In that case, 
Stata just starts at step 1 again, loading mycmd.ado, ...

Just as -mycmd- is guaranteed that when it runs -myfunction()-, it
will be the private -myfunction()- that runs, so it is that other
parts of Stata cannot even access the private -myfunction()-.

When I say that myfunction() is private, I mean it.  In part Stata
(and Mata) arrange for this by never storing myfunction() in the usual
"global" place.  As far as most commands are concerned, -myfunction()-
simply does not exist.

When Laszlo coded -mata mosave myfunction(), replace- in his 
ado-file, -mata mosave- simply could not find -myfunction()-
because nothing named -myfunction()- was saved in the global place. 

When Laszlo reordered his ado-file by not putting the entry point 
(-program mycmd-) first, he confused Stata.  Stata no longer knew 
that -myfunction()- was private, so -myfunction()- was compiled 
into the global space.  There was no point to that, however, 
because -mycmd- would never use the compiled code stored in 
myfunction.mo, now or in the fuure. 

The copy of the compiled code that -mycmd- would use right now is 
the copy in memory, which is admittedly identical to what was 
stored in myfunction.mo.  Future executions of -mycmd-, so long as 
-mycmd- remained in memory, would also use the copy still in memory. 
Future executions of -mycmd-, once the memory copy disappeared, would 
cause execution of -mycmd.ado-, which would recompile myfunction()
(and waste time resaving the file myfunction.mo).

I suspect that Laszlo was hoping that by saving myfunction() in a .mo
file, he would somehow speed future executions.  In fact, he slowed
them down.  He put Mata to the work of writing myfunction.mo, but
there was no corresponding component that would ever read the 
file instead of performing the compile. 

The right approach is simply to include private Mata code in the 
ado-file and let Mata compile the code each time the ado-file is 
loaded.  I write ado-files like that. 

For instance, file rename.ado contains 4,048 lines.  3,930 of those
lines are Mata code that are recomiled everytime rename.ado is 
loaded.  No one has ever complained or even noticed. 

-- Bill
wgould@stata.com
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/