Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Guidance on matrix inversion for OLS in mata
From
Thomas Jacobs <[email protected]>
To
[email protected]
Subject
Re: st: Guidance on matrix inversion for OLS in mata
Date
Tue, 27 Apr 2010 12:12:46 -0500
Antoine,
Thanks again. That worked like a charm. For the future reference of
anyone else going down this path the final solution was
invsym(cross(X,X))*cross(X,Y)
without any primes on the matrices and X already has the column of 1s
included. This produces identical values to regress even if both X
and Y have missing values.
Tom
On Tue, Apr 27, 2010 at 11:34 AM, Thomas Jacobs <[email protected]> wrote:
> Antoine,
>
> Thanks for the reply. I caught my first error but appreciate the
> suggestion as it addresses the missing values issue automatically.
>
> Tom
>
> On Tue, Apr 27, 2010 at 11:29 AM, Antoine Terracol
> <[email protected]> wrote:
>> Hi Thomas,
>>
>> have a look at cross() in the mata help files.
>>
>> invsym(cross(X,X)) produces (X'X)^-1
>>
>>
>> Antoine
>>
>> On 27/04/2010 18:17, Thomas Jacobs wrote:
>>>
>>> Austin,
>>>
>>> Thanks for the reply. I guess I have no choice but to do as you
>>> suggest or else rewrite the program entirely in Stata. I don't have
>>> the days it always takes me to decipher how to do the stata mata data
>>> transfer for this project at present. Every time I sit down to do
>>> something like this I always feel it a roll of the dice. Do I go the
>>> mata route and discover what should have been done in stata or go the
>>> stata route and discover the opposite!
>>>
>>> Here is a short excerpt of the problem. I am simply trying to compute
>>> x'x inverse for a 100 trading day time series of index returns and use
>>> it to solve for betahat for a similar time series of firm returns
>>> where the latter often has one or more missing values.
>>>
>>> Here are the initial steps I take in mata and will use an example with
>>> no missing values:
>>>
>>> X=J(100,2,1)
>>> X[.,2]=LnBAATreasSprd[800::899]
>>> : X
>>> 1 2
>>> +-------------------------------+
>>> 1 | 1 -.005838091 |
>>> 2 | 1 -.0063325767 |
>>> 3 | 1 .0100309728 |
>>> 4 | 1 .0040479107 |
>>> 5 | 1 .000136131 |
>>> 6 | 1 .00040839 |
>>> 7 | 1 -.0148075884 |
>>> 8 | 1 -.0107393684 |
>>> 9 | 1 -.002842935 |
>>> 10 | 1 .0066983248 |
>>> 11 | 1 -.0093153818 |
>>> 12 | 1 -.0502348617 |
>>> 13 | 1 -.0178726967 |
>>> 14 | 1 .0145704737 |
>>> 15 | 1 -.0073333359 |
>>> 16 | 1 -.0029384219 |
>>> 17 | 1 -.0095219389 |
>>> 18 | 1 -.0128216762 |
>>> 19 | 1 -.0150571838 |
>>> 20 | 1 .005833914 |
>>> 21 | 1 .0069765295 |
>>> 22 | 1 .0038266596 |
>>> 23 | 1 -.0067447214 |
>>> 24 | 1 -.0082368711 |
>>> 25 | 1 .0079292208 |
>>> 26 | 1 -.0185827948 |
>>> 27 | 1 -.0081839114 |
>>> 28 | 1 .0103758555 |
>>> 29 | 1 -.007115229 |
>>> 30 | 1 -.0043146866 |
>>> 31 | 1 .0007379653 |
>>> 32 | 1 -.0010016384 |
>>> 33 | 1 .0068858997 |
>>> 34 | 1 -.0044625248 |
>>> 35 | 1 .0031520503 |
>>> 36 | 1 -.0554844476 |
>>> 37 | 1 .0113020828 |
>>> 38 | 1 .0031746107 |
>>> 39 | 1 .0117888227 |
>>> 40 | 1 -.0010267079 |
>>> 41 | 1 .0095236665 |
>>> 42 | 1 .0261100251 |
>>> 43 | 1 -.0126527818 |
>>> 44 | 1 -.0114778923 |
>>> 45 | 1 .0066052717 |
>>> 46 | 1 -.0084775724 |
>>> 47 | 1 -.0032715374 |
>>> 48 | 1 -.0015591006 |
>>> 49 | 1 .0142634539 |
>>> 50 | 1 .0003712231 |
>>> 51 | 1 -.0077179475 |
>>> 52 | 1 .0082479911 |
>>> 53 | 1 .0071813553 |
>>> 54 | 1 .0058756177 |
>>> 55 | 1 -.0082988022 |
>>> 56 | 1 -.0003692649 |
>>> 57 | 1 -.0046538925 |
>>> 58 | 1 -.0049419519 |
>>> 59 | 1 .0040403828 |
>>> 60 | 1 -.0099174296 |
>>> 61 | 1 .0231989622 |
>>> 62 | 1 -.0097859399 |
>>> 63 | 1 -.0028060512 |
>>> 64 | 1 -.0028140107 |
>>> 65 | 1 .0092088645 |
>>> 66 | 1 .0032609063 |
>>> 67 | 1 .0080019441 |
>>> 68 | 1 .0002604046 |
>>> 69 | 1 .0027563504 |
>>> 70 | 1 -.0134371053 |
>>> 71 | 1 .0051452187 |
>>> 72 | 1 .0188821666 |
>>> 73 | 1 .0129679879 |
>>> 74 | 1 .0097926175 |
>>> 75 | 1 .0015559109 |
>>> 76 | 1 -.0006521898 |
>>> 77 | 1 .0055551799 |
>>> 78 | 1 .0016954662 |
>>> 79 | 1 0 |
>>> 80 | 1 -.0011965502 |
>>> 81 | 1 -.0123975417 |
>>> 82 | 1 .0008581794 |
>>> 83 | 1 .0113398973 |
>>> 84 | 1 -.0044504236 |
>>> 85 | 1 -.0101748193 |
>>> 86 | 1 .0006073291 |
>>> 87 | 1 .0028294155 |
>>> 88 | 1 -.0016158026 |
>>> 89 | 1 -.009393123 |
>>> 90 | 1 -.0073215333 |
>>> 91 | 1 .0056367237 |
>>> 92 | 1 -.0044041635 |
>>> 93 | 1 .0046085101 |
>>> 94 | 1 .0162160564 |
>>> 95 | 1 .0047139199 |
>>> 96 | 1 -.0005505134 |
>>> 97 | 1 .0033981025 |
>>> 98 | 1 .034180887 |
>>> 99 | 1 -.0102245966 |
>>> 100 | 1 -.0060580331 |
>>> +-------------------------------+
>>>
>>> and the inverse step with a partial excerpt (all values are zero save
>>> for three cells):
>>> : invsym(X*X')
>>> [symmetric]
>>> 1 2 3 4
>>> 5
>>>
>>> +----------------------------------------------------------------------------
>>> 1 | 0
>>> 2 | 0 0
>>> 3 | 0 0 0
>>> 4 | 0 0 0 0
>>> 5 | 0 0 0 0
>>> 0
>>> 6 | 0 0 0 0
>>> 0
>>> 7 | 0 0 0 0
>>> 0
>>> 8 | 0 0 0 0
>>> 0
>>> 9 | 0 0 0 0
>>> 0
>>> 10 | 0 0 0 0
>>> 0
>>> 11 | 0 0 0 0
>>> 0
>>> 12 | 0 0 0 0
>>> 0
>>> 13 | 0 0 0 0
>>> 0
>>> 14 | 0 0 0 0
>>> 0
>>> 15 | 0 0 0 0
>>> 0
>>> 16 | 0 0 0 0
>>> 0
>>> 17 | 0 0 0 0
>>> 0
>>> 18 | 0 0 0 0
>>> 0
>>> 19 | 0 0 0 0
>>> 0
>>> 20 | 0 0 0 0
>>> 0
>>> 21 | 0 0 0 0
>>> 0
>>> 22 | 0 0 0 0
>>> 0
>>> 23 | 0 0 0 0
>>> 0
>>> 24 | 0 0 0 0
>>> 0
>>> 25 | 0 0 0 0
>>> 0
>>> 26 | 0 0 0 0
>>> 0
>>> 27 | 0 0 0 0
>>> 0
>>> 28 | 0 0 0 0
>>> 0
>>> 29 | 0 0 0 0
>>> 0
>>> 30 | 0 0 0 0
>>> 0
>>> 31 | 0 0 0 0
>>> 0
>>> 32 | 0 0 0 0
>>> 0
>>> 33 | 0 0 0 0
>>> 0
>>> 34 | 0 0 0 0
>>> 0
>>> 35 | 0 0 0 0
>>> 0
>>> 36 | 0 0 0 0
>>> 0
>>> 37 | 0 0 0 0
>>> 0
>>> 38 | 0 0 0 0
>>> 0
>>> 39 | 0 0 0 0
>>> 0
>>> 40 | 0 0 0 0
>>> 0
>>>
>>> qrinv gives something similar with a few more non-zero values while
>>> cholinv and luinv produce nulls. pinv does work in this case.
>>> Suffice it to say I have no problem regressing a similar single firm
>>> return series on this index for the time period in question using
>>> regress.
>>>
>>> If I am missing something, please let me know. Otherwise, I guess I
>>> need to go to stata or do a better job of reproducing regress! Thanks
>>> again.
>>>
>>> Tom
>>> On Tue, Apr 27, 2010 at 10:23 AM, Austin Nichols
>>> <[email protected]> wrote:
>>>>
>>>> Thomas Jacobs<[email protected]>:
>>>> Zero is not a problem, but you should expunge the missings first;
>>>> however you seem to be trying to rewrite -regress- as you go, which is
>>>> far from a good idea. Why *not* export your vectors to Stata and run
>>>> -regress- and let Stata handle the sample selection and matrix
>>>> inversion for you?
>>>>
>>>> Maybe if you give us a simple example with real numbers, the problem
>>>> will be clearer and you can get better guidance...
>>>>
>>>> On Tue, Apr 27, 2010 at 12:45 AM, Thomas Jacobs<[email protected]>
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I am trying to perform a lengthy series of simulations to examine some
>>>>> event study methodologies. I have moved to mata for the bulk of the
>>>>> work but find that for those cases where I wish to use a market model
>>>>> approach requiring an OLS regression to establish abnormal returns I
>>>>> am unable to generate an inverse for x'x in seeking to solve for beta
>>>>> hat. I am typically working with vectors that have 1. missing values,
>>>>> 2. zero values, and 3. very small values close to zero (within a
>>>>> couple of decimal places such as -.01 or .005). I have tried mata's
>>>>> cholinv, invsym, pinv, luinv, and qrinv (I realize that some of these
>>>>> are probably inappropriate for my problem but I am no expert) and
>>>>> generally get an inverse matrix of missing values or bizarre results
>>>>> like a single populated row.
>>>>>
>>>>> I would prefer not to go back and forth between stata and mata to use
>>>>> the stata regress function unless that is the only way to accomplish
>>>>> this effort.
>>>>>
>>>>> Can anyone offer general guidance on how to proceed here? Thanks.
>>>>>
>>>>> Tom
>>>>
>>>> *
>>>> * For searches and help try:
>>>> * http://www.stata.com/help.cgi?search
>>>> * http://www.stata.com/support/statalist/faq
>>>> * http://www.ats.ucla.edu/stat/stata/
>>>
>>>
>>>
>>> --
>>> Thomas Jacobs
>>>
>>> *
>>> * For searches and help try:
>>> * http://www.stata.com/help.cgi?search
>>> * http://www.stata.com/support/statalist/faq
>>> * http://www.ats.ucla.edu/stat/stata/
>>>
>>
>> --
>> Ce message a ete verifie par MailScanner
>> pour des virus ou des polluriels et rien de
>> suspect n'a ete trouve.
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>>
>
>
>
> --
> Thomas Jacobs
>
--
Thomas Jacobs
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/