One subtlety when parsing non-standard syntax is the following.
Suppose you're parsing on ".", and you run into two or more of these
characters together. In this case, -gettoken- does the following:
. loc foo "..."
. gettoken tok foo : foo, parse(".")
. di "|`tok'|"
|.|
. gettoken tok foo : foo, parse(".")
. di "|`tok'|"
|.|
. gettoken tok foo : foo, parse(".")
. di "|`tok'|"
|.|
By contrast, suppose you're parsing on "=":
. loc foo "==="
. gettoken tok foo : foo, parse("=")
. di "|`tok'|"
|==|
. gettoken tok foo : foo, parse("=")
. di "|`tok'|"
|=|
In other words, when parsing on "=" -gettoken- treats "==" as though
it were "=". It's not hard to image how this behavior might be
useful, given the special status of "==" as a relational operator.
Still, unless I missed it, this special treatment of "=" is not
mentioned in [P] gettoken. I haven't had a need to change this
behavior, so I'm not asking for a "treat '=' as other characters"
option on -gettoken-. But a sentence in the manual would save me
from having to figure this out again each time I run across it ;)
