Re: st: ucm with daily data

[Sergiy Radyakin <serjradyakin@gmail.com>]

Dear Dimitriy, thank you for posting this info back to the thread. I
didn't know about this dimensionality problem. It seems reasonable to
be more cautious about the choice of t then. One advice (and it
depends on the nature of your phenomenon under study, is that perhaps
you could reduce from 365(=52*7) to 310(=52*6) by counting the
weekends as one long day. Not sure if your process is sensitive to
that, but if nothing happens on the weekend (a la stocks not traded)
than there is no point to split the weekend into Saturday and Sunday,
but one can count it as one (longer) day. Again this probably does not
make a huge difference (365^4/310^4=1.92) and might not be applicable,
but who knows.

is the quoted dimension t^2+t^4 the side of the matrix or number of
elements?(both are huge of course)

Best, Sergiy


On Wed, Jul 3, 2013 at 8:10 PM, Dimitriy V. Masterov <dvmaster@gmail.com> wrote:
> The main issues is that ucm requires computing a Kronecker product
> that generates a matrix with a dimension larger than t^2+t^4, which is
> rather large for t=365. This is the reason why the limit was set to
> t=52, which represents a weekly seasonal pattern. Stata developers are
> looking into raising this limit to something loftier. In the mean
> time, they suggested aggregating to weekly and using 52.
>
> I took Sergiy's intrepid advice and "created" ucm365.ado by changing
> the definition of local m to -local m = min(365,`N')-. It's been
> searching for initial values for most of today, but it seems to be
> running. I'll report back if this actually works.
>
> DVM
>
> On Wed, Jun 26, 2013 at 12:28 PM, Nick Cox <njcoxstata@gmail.com> wrote:
>> I doubt Dimitriy needs this comment, but for any newcomers listening,
>> the standard and longstanding advice here is to clone the file under a
>> new name and then change it, taking full responsibility for the change
>> and its effects.
>>
>> An important detailed change with Stata 13 is that updated files will
>> overwrite existing files, which adds extra point to that advice. If
>> you change -foo.ado- to your liking, not only is that a bad idea, the
>> next time Stata updates -foo.ado- that process will overwrite your
>> copy.
>> Nick
>> njcoxstata@gmail.com
>>
>>
>> On 26 June 2013 20:22, Sergiy Radyakin <serjradyakin@gmail.com> wrote:
>>> Dimitriy,
>>>
>>> the check against 52 is implemented around line 203 in ucm.ado. This
>>> looks to me like an artificial safety measure. The rest of the program
>>> refers to 365, so it might work if you simply disable the check. You
>>> may encounter another problem later on then (and whether you encounter
>>> it immediately or later might be data dependent), plus your program
>>> will not work on other computers with Stata since they will contain
>>> the original version of ucm.ado.
>>>
>>> ucm.ado is by StataCorp. You will need to consult with them whether
>>> there is anything (technical or legal) that can prevent you from doing
>>> such modifications. They might also shed some light onto why the
>>> condition is there in the first place.
>>>
>>> Best,
>>>    Sergiy Radyakin
>>>
>>>
>>>
>>>
>>> On Wed, Jun 26, 2013 at 12:45 PM, Dimitriy V. Masterov
>>> <dvmaster@gmail.com> wrote:
>>>> Sergiy,
>>>>
>>>> Sorry for the confusion. UCM stands for unobserved components model.
>>>> It decomposes a time series into trend, seasonality, an idiosyncratic
>>>> component, and up to 3 cycle(s) using a Kaman filter state space
>>>> approach with the parameter estimated by MLE. It's a native Stata
>>>> command, like ARIMA, and it allows for exogenous variables. I think
>>>> UCM is standard terminology for this sort of structural time series
>>>> model. For instance, SAS has a proc ucm.
>>>>
>>>> Seasonality is specified as an option: ucm bi, seasonal(365)
>>>>
>>>> The number in the parentheses is the period of the season (the number
>>>> of time-series observations required for the period to complete).
>>>> Stata complains that 365 exceeds 52, but I can't find this limitation
>>>> detailed anywhere. I have daily data, so weekly seasonality seems like
>>>> a tight constraint.
>>>>
>>>> The exact error messages is:
>>>>
>>>> seasonal(365) exceeds 52; this is not allowed
>>>> r(459);
>>>>
>>>>
>>>> DVM
>>>>
>>>> On Wed, Jun 26, 2013 at 6:44 AM, Sergiy Radyakin <serjradyakin@gmail.com> wrote:
>>>>> Dimitriy, once you tell us what UCM is and what command you are
>>>>> running, it would be possible to look at the source and see whether
>>>>> the constraint is serious and necessary for the program to work, or
>>>>> just a safety constraint that can be relaxed by modifying the
>>>>> threshold value.Sergiy
>>>>>
>>>>> On Tue, Jun 25, 2013 at 11:19 PM, Dimitriy V. Masterov
>>>>> <dvmaster@gmail.com> wrote:
>>>>>> I am trying to fit a UCM with a daily time series. Seasonal(365)
>>>>>> throws a r(459) error, and tells me not to exceed 52. This limitation
>>>>>> is not described anywhere.
>>>>>>
>>>>>> Any ideas on how to deal with this problem?
>>>>>>
>>>>>> DVM
>>>>>> *
>>>>>> *   For searches and help try:
>>>>>> *   http://www.stata.com/help.cgi?search
>>>>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>>>>> *   http://www.ats.ucla.edu/stat/stata/
>>>>> *
>>>>> *   For searches and help try:
>>>>> *   http://www.stata.com/help.cgi?search
>>>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>>>> *   http://www.ats.ucla.edu/stat/stata/
>>>> *
>>>> *   For searches and help try:
>>>> *   http://www.stata.com/help.cgi?search
>>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>>> *   http://www.ats.ucla.edu/stat/stata/
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>> *   http://www.ats.ucla.edu/stat/stata/
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/