Yesterday I replied to a question from Antoine Terracol
<[email protected]>, the first part which was about multiplying a
3,188,029 x 22 matrix by a 22 x 22 matrix, x*u. Matrix x was a view.
Antoine had used cross,
x = cross(x', u)
but ran ouit of memory and I explained that was caused by the x', and said
better would be
x = x * u
although I was still negative about the whole thing working out because
even my suggestion would allocate a 535M matrix to store the result
before assignment back to x.
Pretend there was sufficient memory. There is still a problem in what I
suggested because x is a view matrix. I should have suggested
x[.,.] = x * u
and, by the same token, had there been sufficient memory for Antoine's
original solution, he should have coded
x[.,.] = cross(x', u)
The problem is that statements like
x = cross(x', u)
and
x = x*u
replce the definition of x, and what we want to do is maintain x as a
view matrix and replace its elements.
I know this sounds like a fine distinction, so consider another, easier case.
In some other problem, pretend we had matrix X and, to keep it simple, X is
not even a view matrix. It is just a regular Mata matrix. Let's pretend that
X, right now, is 3 x 4. If we were to code
X = Y
and if Y were 20 x 2, we would not expect the line to generate an error, and
it does not. It does not generate an error because the definition on the
right REPLACES the definition on the left. Old 3 x 4 matrix X is discarded
and it sreplaced with a brand new, 20 x 2 one. On the other hand, were we to
code
X[,.,] = Y
we would expect an error, because that line says to replace the elements of X,
and existing 3 x 4 matrixX cannot hold a 20 x 2 result.
Usually this fine distinciton never matters, and we code X = Y without
thinking. It does, however, matter with view matrices. Going back to
Antoine's problem, code
x = x * u
and you are saying to replace the matrix on the left. Eliminate existing
matrix x, which happens to be a view, and create a shiny new matrix
conformable with the result x*u. In this case, however, Antoine wants to
maintain the existing definition of x and merely replace its elements. x is a
view matrix and changing its elements is what changes the underlying Stata
dataset.
I'm done with that.
In any case, yesterday I was negative about
x[.,.] = x*u
working ouit for Antoine because of lack of memory, and I suggested a
a partitioned solution. That part was correct, but even easier might
simply be to code
for (i=1; i<=rows(x); i++) {
x[i,.] = x[i,.] * u
}
It won't be as fast, but it certainly is easy to code. This is, of course,
merely an extreme example of the partioned solution.
-- Bill
[email protected]
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
