st: Re: st_views passed on from Mata funtion to Mata function? pointers?

[László Sándor <sandorl@gmail.com>]

FWIW, I now checked lars.ado, and there are operations on its st_view,
even the one definitely avoidable with cross(). So presumably it is
not much more efficient, at least memorywise.

On Wed, Sep 11, 2013 at 2:43 PM, László Sándor <sandorl@gmail.com> wrote:
> Hi,
> I am still struggling to make Christian Hansen's lassoShooting as fast
> as possible. (See yesterday's email, code at
> http://faculty.chicagobooth.edu/christian.hansen/research/)
>
> Even if I don't run out of memory (it is already scary if my data
> takes more than 50% of physical memory so a copy runs out of it), I am
> worried about lines of code when a view is passed onto a function that
> expects a real matrix. A copy is made there, inefficiently?
>
> See e.g. the function
>
> real rowvector MakeLassoWeights(real colvector v , real matrix XX ,
> real scalar hetero )
> {
> real rowvector Ups
> nObs = rows(v)
> if (hetero == 1) {
> St = XX:*(v*J(1,cols(XX),1))
> Ups = sqrt(quadcolsum((St):^2)/nObs)
> }
> else {
> Ups = sqrt(quadcolsum(XX:^2)/nObs)*sqrt(quadcolsum(v:^2)/nObs)
> }
> return(Ups)
> }
>
> And its use
> Ups = MakeLassoWeights(v , data.Xs , hetero)
>
> Where data.Xs is a HUGE view (think of 3 million rows, 500 columns),
> and even v is long. Btw, I still think v should made more efficiently:
> v = data.y - data.Xs*betas.betaAll
>
> I find it stunning if Stata proudly sports a cross() function proudly
> careful and efficient with views but does not do something similar for
> other matrix multiplications.
>
> Is there really nothing to do about this?
>
> Or has anyone vetted LARS.ado much to see if their use of Mata is more
> careful? (I cannot try the new code easily with the data.)
>
> Thanks,
>
> Laszlo

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/