8.77. File tom/XL

class tom.XLTokens

The tokens available from XL.

variables

const XLT_PAR_CLOSE = -9;

const XLT_PAR_OPEN = -8;

const XLT_DOUBLE = -7;

const XLT_FLOAT = -6;

const XLT_LONG = -5;

const XLT_INT = -4;

const XLT_SYMBOL = -3;

const XLT_EPSILON = -2;

const XLT_EOF = -1;

instance tom.XLTokens

class tom.XL

inherits

State supers: State, XLTokens

variables

const XLS_SYMBOL = 0;

Different states of the lexer state machine. Basically, these states are the states of reading a floating point number, with a prefix for an integer, and an escape for a non-numeric input.

const XLS_SIGN = 1;

const XLS_INT = 2;

const XLS_DOT = 3;

const XLS_FRAC = 4;

const XLS_EXP_E = 5;

const XLS_EXP_SIGN = 6;

const XLS_EXP = 7;

instance tom.XL

variables

public InputStream stream;

The stream being lexed.

MutableByteString buffer;

The buffer used for building the text of the token.

public long int_value;

The most recent integer value retrieved.

public double float_value;

The most recent floating value retrieved.

public int current_line;

The current line.

public int token;

The current token.

int next_char;

The next character, i.e. the first character of the next token. This is XLT_EOF for end of stream, or XLT_EPSILON if this should be considered invalid (and read before starting the next token).

methods

id
  initWithStream InputStream s;

Designated initializer.

int
  intValue;

Return the int_value as an int. Any loss of bits is not remarked.

MutableString
  matched;

Return the matched text.

int
  nextToken;

Skip space and return the next token.

(pointer, int) (contents, length)
  readBytes int expected_length
post
  length == expected_length;

Skip whitespace, read a quoted string of bytes ("quoting \\like\\ \"this\"") and return it. The length should match the expected length. Anything unexpected results in the return of a NULL pointer.