st: Mata use of memory for data -- duplicate memory allocation, multiple copies?

[László Sándor <sandorl@gmail.com>]

I am trying to formulate my previous concerns into a more fruitful
discussion for all here. Maybe distinguished Statalist members with
Mata coding experience could share their experiences.

I think my biggest question is that if it is this hard to use st_views
without making a copy of the data in anything apart from the cross()
function, does it go without saying that all programs written in Mata
will take memory on the order of magnitude of the data you use? This
not only depletes system resources but also take plenty of copying,
e.g. if the copy is made again and again for all iterations of an
estimator.

If this is the case, I wonder why I found no more prominent resources
about the issue. Maybe even Bill could have mentioned this in the
Missing Manual and blogposts, or Kit in ITSP.

Also, this sheds new light on the StataCorp hardware recommendation
that generally you want to have 50% more RAM than the largest data you
would ever use.

If I am missing something here, I am happy to learn!

Thanks again,

Laszlo
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/