Hi Lina,
type:
viewsource spatwmat.ado
in Stata's command prompt to see the source code
Also you can:
set trace on
before calling spatwmat and see where it breaks down.
Best regards, Sergiy
On 9/26/07, Lina Jonsson <[email protected]> wrote:
> Hi again.
>
>
> Thank you for the quick reply Sergiy!
>
> There is no problem running the code you gave me. I belive that the spatwmat-command is creating something temporarily that takes a lot of memory. I have only been using Stata for a couple of months so I have never looked into a source code before - how do I do that?
>
> Does anyone else have experience in using the spatwmat-command on "large" (more than 800 observations) datasets?
>
> Best Regards
>
> /Lina Jonsson
>
> > -----Ursprungligt meddelande-----
> > Fr�n: [email protected]
> > [mailto:[email protected]] F�r Sergiy Radyakin
> > Skickat: den 26 september 2007 12:34
> > Till: [email protected]
> > �mne: Re: st: SV: Re: Memory problems when creating a spatial
> > weight matrix
> >
> > Hi Lina,
> >
> > may be I am missing something, but a matrix 500x500 at 4
> > bytes per cell occupies 1MB of memory. At 8 bytes per cell
> > 2MB respectively. How does this relate to the 500/900MB of
> > memory limit? As I understand, your dataset is 800
> > observations and -spatwmat- produces a matrix 800x800.
> >
> > Could you run the following and tell us what happens?
> >
> > clear
> > set mem 100m <-- or whatever your maximum is
> > set matsize 800
> > matrix Z=J(800,800,0)
> > matrix dir
> > matrix A=J(800,800,0)
> > matrix dir
> > matrix B=J(800,800,0)
> > matrix dir
> > matrix C=J(800,800,0)
> > matrix dir
> > use "mydata" <--- whatever your dataset is
> >
> > If it works fine, than it is not a memory issue, but rather
> > you'd have to look into the source code of -spatwmat- to find
> > if the memory is used there efficiently (e.g. it does not
> > generate 800 temporary matrices).
> >
> > From the first look, it might be relatively easy to port the
> > code into matlab. However then you need matlab, why not Stata
> > SE? If you don't have constraints with software -- buy Stata
> > 10 SE and this might resolve the issue. If you do have
> > constraints, than what are they (which software is available
> > to you)? There might be just a handful of people working with
> > specialized commands like this, so please explain us what
> > this command does in plain English.
> >
> > Best regards,
> > Sergiy Radyakin
> >
> >
> >
> >
> >
> > On 9/26/07, Lina Jonsson <[email protected]> wrote:
> > > Dear statalister.
> > >
> > > Sorry I haven�t had time to follow up my question until now.
> > >
> > > The memory problem occurs only when using the command for
> > creating the spatial weight matrix (spatwmat). The dataset
> > I�m using consists of only 28 variables - all numerical and
> > dummies (0,1). According to an earlier question concering
> > memory problems when using the spatwmat-command; "-spatwmat-
> > will reset the number of observations to (N/2)*N-1" which
> > means about 320 000 if I�m using a dataset with 800
> > observations. The advice that is given in the former question
> > is to set memory to a higher value. That is the advice that
> > also is given for my error message "no room to add more
> > variables". But the problem is that I cannot set memory to
> > more then 940m.
> > >
> > > I am using Windows XP and have just increased my RAM-memory
> > to 2 GB which had no effect at all on the maximum memory I
> > can set within Stata. As I understand it; the hotfix Michel
> > mentioned is only for Stata 10 and I`m using Intercooled Stata 9.2.
> > >
> > > Is anyone having any suggestions on how this problem can be
> > solved or is it not possible to use the spatwmat-command on
> > datasets with 800 observation using "a normal computer". In
> > fact I whould like to use up to 1738 observations but for
> > that I whould need to switch to Stata SE instead of
> > Intercooled Stata due to the fact that Intercooled Stata can
> > set matsize to a maximum of 800. But there is no point in
> > changing to Stata SE if the available memory sets the limit
> > to less than 800 observations...
> > >
> > > If Stata can�t handle spatial weight matrices for large
> > datasets, do any of you have any suggestions of other
> > statistical software that can?
> > >
> > > Best regards.
> > >
> > > /Lina Jonsson
> > >
> > >
> > >
> > >
> > > > Fr�n: [email protected]
> > > > [mailto:[email protected]] F�r Michael Blasnik
> > > > Skickat: den 21 september 2007 14:49
> > > > Till: [email protected]
> > > > �mne: st: Re: Memory problems when creating a spatial
> > weight matrix
> > > >
> > > > You need to give us more information about your dataset,
> > operating
> > > > system, etc.
> > > > The 940m should be plenty of room for a 500 observation
> > dataset and
> > > > an 800x800 matrix. Can you describe your dataset in more
> > detail?
> > > > Do you have hundreds of long string variables? Is there
> > a way for
> > > > you to encode any long strings and compress other variables? In
> > > > terms of accessing more memory -- what OS are you using? If MS
> > > > Windows XP, have you tried the memory hotfix described here
> > > > http://www.stata.com/support/faqs/win/winmemory.html?
> > > >
> > > > Michael Blasnik
> > > >
> > > > ----- Original Message -----
> > > > From: "Lina Jonsson" <[email protected]>
> > > > To: <[email protected]>
> > > > Sent: Friday, September 21, 2007 5:25 AM
> > > > Subject: st: Memory problems when creating a spatial weight matrix
> > > >
> > > >
> > > > > Dear statalisters.
> > > > >
> > > > > I am working with a hedonic price study where I whould like
> > > > to check for
> > > > > and correct for spatial autocorrelation. For this I
> > whould like to
> > > > > create a spatial weight matrix based on the x- and
> > y-coordinates
> > > > > for each house using the spatwmat-command in spdep. Since I
> > > > have Intercooled
> > > > > Stata the maximum matsize that I can set is 800, which
> > > > means that I can
> > > > > use no more than 800 observations. But since I cannot set
> > > > memory to more
> > > > > than 940m, the actual numbers of observations that I can use is
> > > > > less than 500. This is to few.
> > > > >
> > > > > After reading some old questions concerning memory problems
> > > > I realize
> > > > > that this might be a problem that is hard to solve.
> > > > > Does any of you have any suggestions on how I can increase
> > > > the memory in
> > > > > Stata. Can increasing the RAM-memory on my computer
> > help? Or is it
> > > > > unrealistic to think that it will be possible to create a
> > > > matrix whith
> > > > > the spatwmat-command that will be almost 8000x800 on a
> > "standard"
> > > > > computer? Should I change to some other software and what
> > > > in that case?
> > > > >
> > > > > I am greatful for any suggestions!
> > > > >
> > > > > /Lina Jonsson
> > > >
> > > > *
> > > > * For searches and help try:
> > > > * http://www.stata.com/support/faqs/res/findit.html
> > > > * http://www.stata.com/support/statalist/faq
> > > > * http://www.ats.ucla.edu/stat/stata/
> > > >
> > >
> > > *
> > > * For searches and help try:
> > > * http://www.stata.com/support/faqs/res/findit.html
> > > * http://www.stata.com/support/statalist/faq
> > > * http://www.ats.ucla.edu/stat/stata/
> > >
> >
> > *
> > * For searches and help try:
> > * http://www.stata.com/support/faqs/res/findit.html
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
>
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
