API documentation: internals

collecttail(s, arr)
collecttail(s, arr, stripped)

Remove all trailing characters belong to list in arr, and collect them. This is useful to collect punctuation from a token, for example.

Remove all trailing characters belong to list in arr, and return the trimmed string.

trimtail(s, arr)

This is useful to remove punctuation from a token, for example.

Compose list of alphabetic characters in SimpleAscii.

asciialphabetic()

Compose list of numeric characters in SimpleAscii.

asciinumeric()

Allow notation of positive or negative decimals.

Compose list of punctuation characters in SimpleAscii.

asciipunctuation()

Compose list of recognized whitespace characters in SimpleAscii.

asciiwhitespace()

Define a string including all valid code points in the SimpleAscii orthography.

asciiCPs()

Define an Array with all valid TokenCategorysystems in the SimpleAscii orthography.

basicTypes()

Split off any trailing punctuation and return an Array of leading strim + trailing punctuation.

splitAsciiPunctuation(s)

Construct an OrthographicToken from a string.

asciitokenforstring(s)

s is a string representing a single token in the SimpleAscii orthography.

True if all characters in s are alphabetic.

isAsciiAlphabetic(s)

True if all characters in s are punctuation.

isAsciiPunctuation(s)

True if all characters in s are numeric characters and at least one is a digits.

isAsciiNumeric(s)

Obviously not a real-world definition of syntax for a numeric token.