Re: st: Twitter Message Sub-string Extraction?

[Eric Booth <ebooth@ppri.tamu.edu>]

<>
On Jun 13, 2011, at 2:56 PM, Richard Fairbanks wrote:
<snip>
> I still have one issue due to another data element I thought I could
> handle. The tweets also contain URLs. They're a mix of letters and
> integers. Even using the same loops you provide doesn't work for them
> (i.e., replacing things like "@" and "#" with "http:\\" won't pick out
> the full web address.

Keep in mind that you can use things like "http:\\" or "ftp:\\" in the strpos()/subinstr() commands in the previous examples. E.g., 
*****
clear
input str129(test)
"test #testt http://www.google.com/_test blah"
end
split test
foreach v in varlist test* {
foreach j in file ftp http {
	replace url = `v' if strpos(`v', "`j'://")
}
*****

I'm not a regex-pert ... but maybe you could get a Stata-friendly version of one of these to work:

http://www.regexguru.com/2008/11/detecting-urls-in-a-block-of-text/
http://blogs.lse.ac.uk/clt/2008/04/23/a-regular-expression-to-match-any-url/

I'd be interested in hearing back if you come up with a good statement to match urls.
I've got one that worked pretty well with matching email addresses if that is a concern as well 
	-- it's here on my webpage under the -validemail.ado- section:  http://goo.gl/QIr2y


- Eric
__
Eric A. Booth
Public Policy Research Institute
Texas A&M University
ebooth@ppri.tamu.edu





*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/