Lexical Conventions

Lexical Conventions#

Warning

This page is a work in progress and is currently incomplete.

This page specifies MoonBit lexical forms. Runtime representation, APIs, and literal overloading are covered in Fundamentals.

In the productions, { symbol } means zero or more repetitions, { symbol }+ means one or more repetitions, and x ... y denotes an inclusive range.

Common Lexical Classes#

hex-digit ::= 0 ... 9 | A ... F | a ... f

octal-digit ::= 0 ... 7

unicode-scalar-value ::= U+0000 ... U+D7FF | U+E000 ... U+10FFFF

newline ::= LF | CR | CR LF | U+2028 | U+2029

whitespace ::= U+0009 | U+000B | U+000C | U+0020 | U+00A0 | U+1680
             | U+2000 ... U+200A | U+202F | U+205F | U+3000 | U+FEFF

String Literals#

string-literal ::= " { string-character } "

string-character ::= regular-string-character
                   | simple-escape-sequence
                   | unicode-escape-sequence
                   | interpolation

regular-string-character ::= unicode-scalar-value except ", \, CR, and LF

simple-escape-sequence ::= \ (\ | " | ' | n | t | b | r | f | /)

unicode-escape-sequence ::= \u hex-digit hex-digit hex-digit hex-digit
                          | \u{ { hex-digit }+ }

The simple escape sequences have the following meanings:

Sequence

Character

\\

Backslash (U+005C)

\"

Double quote (U+0022)

\'

Single quote (U+0027)

\/

Forward slash (U+002F)

\n

Line feed (U+000A)

\r

Carriage return (U+000D)

\t

Horizontal tab (U+0009)

\b

Backspace (U+0008)

\f

Form feed (U+000C)

A Unicode escape must denote a Unicode scalar value. A CR or LF before the closing quote reports an unterminated string literal.

Interpolation#

interpolation ::= \{ { whitespace } expression { whitespace } }

The expression must be nonempty and end at the matching }. Braces inside nested literals do not affect matching; nested interpolations are recognized recursively. CR, LF, // comments, attributes, and multiline string literals are not permitted.

Multiline String Literals#

multiline-string-literal ::= raw-multiline-string-literal
                           | interpolated-multiline-string-literal

raw-multiline-string-literal ::= raw-multiline-string-line
                               { newline raw-multiline-string-line }

interpolated-multiline-string-literal ::= interpolated-multiline-string-line
                                        { newline interpolated-multiline-string-line }

raw-multiline-string-line ::= #| { multiline-regular-character }

interpolated-multiline-string-line ::= $| { multiline-regular-character | interpolation }

multiline-regular-character ::= unicode-scalar-value except CR and LF

The prefixes are omitted from the result, and lines are joined with U+000A. A final empty prefixed line adds a trailing line feed. A #| line is literal; in a $| line, only \{ begins interpolation. Multiline strings are not permitted inside interpolation expressions.

Bytes Literals#

bytes-literal ::= b" { bytes-character } "

bytes-character ::= regular-string-character
                  | simple-escape-sequence
                  | byte-escape-sequence
                  | interpolation

byte-escape-sequence ::= \x hex-digit hex-digit
                       | \o (0 ... 3) octal-digit octal-digit

A CR or LF before the closing quote reports an unterminated literal. Non-ASCII source characters contribute their UTF-8 encoding; \xHH and \oDDD each contribute one byte with a value from 0 to 255. Interpolation follows the string-literal rules. There is no multiline bytes-literal form.

Character Literals#

character-literal ::= ' regular-character '
                    | ' character-escape-sequence '

regular-character ::= unicode-scalar-value except ', \, CR, and LF

character-escape-sequence ::= simple-escape-sequence
                            | unicode-escape-sequence

A character literal contains exactly one Unicode scalar value or escape sequence.

Byte Literals#

byte-literal ::= b' regular-byte-character '
               | b' byte-character-escape-sequence '

regular-byte-character ::= U+0000 ... U+007F except ', \, CR, and LF

byte-character-escape-sequence ::= simple-escape-sequence
                                 | byte-escape-sequence

An unescaped byte is ASCII. Unicode escapes are invalid in byte literals.

Contents