UTF-8 code points, non-owning byte views, owned mutable strings, decoding, and text formatting helpers.

Char

A Unicode code-point value.

Char

(-- char) Creates a Char with code point 0.

Fields

equal

(other -- cond) Compares char with another Char or the first Char decoded from Text.
  • A non-Text operand must provide an Int32 codepoint. A Text operand is converted with toChar; it need not contain exactly one character.

greater

(other -- cond) Returns TRUE when the other Char, or the first Char decoded from Text, has a greater code point than char.
  • A non-Text operand must provide an Int32 codepoint. A Text operand is converted with toChar; it need not contain exactly one character.

hash

(-- hash) Returns the code point cast to Nat32.

less

(other -- cond) Returns TRUE when the other Char, or the first Char decoded from Text, has a lower code point than char.
  • A non-Text operand must provide an Int32 codepoint. A Text operand is converted with toChar; it need not contain exactly one character.

Examples

"String"  use
"control" use

{} () {} [
  65i32 toChar .codepoint printStack _:;
] "main" exportFunction

Expected Output During Compilation

65 Cref

StringView

StringView stores an immutable byte reference and a copied byte count. Its default value has a NIL reference and size zero. Changes to the bytes are visible through a view, but its size does not track its source. Reallocating or freeing a buffer invalidates earlier references, iterators, Spans, and StringViews into it; none keeps the owner alive.

StringView

(-- view) Creates an empty StringView with a zero byte count.
  • The stored Nat8 reference is NIL.

Fields

data

(-- data) Returns an immutable Nat8 reference to the first byte of view.
  • The reference's target has schema Nat8; an empty view can carry its supplied address, while the default view has a NIL reference.

size

(-- size) Returns the view length in UTF-8 code units, that is, bytes.

hash

(-- hash) Hashes each byte with the recurrence result = result * 47 + byte, starting at 33.
  • The result is Nat32 arithmetic.

iter

(-- iter) Returns a byte iterator over the view.
  • next returns an immutable Nat8 reference followed by a Cond. TRUE indicates a byte; FALSE indicates exhaustion, and its reference must not be dereferenced. Unless exhaustion is known, call next through a mutable iterator such as @it.next; an immutable iterator is rejected at compile time. A known-exhausted immutable iterator is accepted and returns its immutable Nat8 Cref with FALSE; the reference may be NIL.

slice

(offset size -- view) Returns a view at byte offset with the supplied byte size.
  • The result refers to the same storage. Offset and size are not bounds-checked here. The result permits immutable byte access, even from a mutable receiver.

span

(-- span) Returns a Span carrying an immutable Nat8 reference and the view's byte count.

Constructors and converters

toStringView

((Nat8 Cref Int32) -- view) Constructs a StringView from a two-item Tuple containing an immutable Nat8 reference and an in-place Int32 byte count. Copy a count reference with new before placing it in the Tuple. The constructor checks the schema, not the address or byte range.
  • A mismatched schema raises [toStringView], invalid argument, (Nat8 Cref Int32) expected at compile time.

makeStringView

(object -- view) Creates a StringView from a StringView, Text, or String object.
  • A StringView is copied as a view; Text and String inputs supply their byte address and byte count. The result remains non-owning.

makeStringViewByAddress

(address -- view) Uses C strlen at address and returns a view of the bytes before the first zero byte.
  • The byte contents are not UTF-8 validated. address has schema Natx and must point to readable zero-terminated storage. The length before the zero must fit in Int32, and the storage must outlive the view.

Examples

"String"  use
"control" use

{} () {} [
  view: "hi" makeStringView;
  view.size printStack _:;
] "main" exportFunction

Expected Output During Compilation

2

String

An owned mutable byte string. Raw byte operations do not enforce well-formed UTF-8. The logical size excludes its trailing zero byte.

String

(-- string) Creates an empty owned String.
  • Its backing Nat8 array starts with size and capacity zero.

Fields

A non-empty backing array contains the logical bytes followed by one zero byte; an empty String need not contain a terminator. Direct changes to chars must preserve this invariant. Destruction frees the buffer still owned by the String.

ASSIGN: assigning another String: an immutable source is copied; a mutable source is moved and becomes empty. Initialization and destruction are automatic; INIT and DIE are reserved lifecycle hooks, not ordinary public API operations.

The cat*NZ methods append to raw chars storage; they neither remove an existing terminator nor add a new one. For an existing String, copy any source that aliases it before calling makeNZ, perform the NZ appends, then call makeZ once. For example, @s.makeNZ "text" @s.catStringNZ @s.makeZ appends to an existing String. Until makeZ restores the invariant, do not use the receiver's logical String operations; String.size excludes its last raw byte. Use chars.size for the raw byte count during this phase.

data

(-- data) Returns a Nat8 reference to the first byte.
  • The reference is mutable through a mutable String receiver and immutable otherwise. It may be NIL before allocation.

hash

(-- hash) Returns the StringView byte hash of the logical string contents.

iter

(-- iter) Returns an iterator over the logical bytes.
  • next requires a mutable iterator unless exhaustion is known; for a nonempty source use @it.next. An immutable iterator over a known-exhausted source is accepted and returns its immutable Nat8 Cref with FALSE; the reference may be NIL. When the Cond is TRUE, the returned reference is mutable when the iterator was created from a mutable String such as @s.iter and immutable otherwise. Ignore the reference when the Cond is FALSE.

size

(-- size) Returns the logical byte count, excluding the trailing zero terminator.

slice

(offset size -- view) Returns a non-owning StringView over the selected byte range.
  • Offset and size are used for address arithmetic without bounds checks; the returned view does not keep the String alive. The result permits immutable byte access, even from a mutable receiver.

catAsciiSymbolCode

(codepoint --) Appends an ASCII code in Nat32 form, from 0 through 127, and restores the trailing terminator.
  • The input must be Nat32. Assertion checking reports Is not ascii symbol code! for an invalid value; disabling runtime assertions does not remove this precondition.

catAsciiSymbolCodeNZ

(codepoint --) Appends an ASCII code point as one byte without adding the String's trailing zero terminator.
  • codepoint is a Nat32 value below 128. Assertion checking reports Is not ascii symbol code! for an invalid value; run-time checking requires DEBUG. Disabling checks does not remove the range precondition.
  • Follow the NZ phase protocol.

catChar

(char --) Accepts Char or another object providing a codepoint convertible to Nat32, and appends that code point as UTF-8. The converted code point must be a Unicode scalar value; this method does not validate it.

catCharNZ

(char --) Appends the input code point encoded as UTF-8 without adding the String's trailing zero terminator.
  • The input supplies a codepoint field that is cast to Nat32; this method does not validate the Unicode scalar range.
  • Follow the NZ phase protocol.

catString

(object --) Appends Text, StringView, or String contents and restores the terminator.
  • A source that aliases the receiver must be copied first (see the known-issue note below).

Known issue: a source that aliases the receiver is read after the receiver has been enlarged, so appending a String, or a view of it, to itself yields undefined bytes; copy the source first: s toString @s.catString.

catStringNZ

(source --) Appends the bytes of a Text, StringView, or String source without adding the String's trailing zero terminator.
  • An empty source appends nothing.
  • Copy a source that aliases the receiver before entering the NZ phase; enlarging the backing array can invalidate a borrowed source.
  • Follow the NZ phase protocol.

catInt

(number by3 --) Appends base-10 text for an integer or real number and restores the terminator.
  • Real inputs must be finite and fit the integer conversion: Int64 for negative inputs and Nat64 otherwise. They lose their fractional part; a negative input retains its minus sign, so -0.5 produces -0. Zero produces 0.
  • by3 is a Cond. FALSE disables grouping; TRUE enables comma grouping; by3 inserts commas between groups of three integer digits. Real-number formatting by catFloat or grouped catMany has the known issue described below.

catIntNZ

(number by3 --) Appends base-10 text for an integer or real number with optional groups of three digits without adding the String's trailing zero terminator.
  • by3 is a Cond; TRUE inserts comma separators and FALSE disables grouping.
  • The numeric conversion and grouping rules of catInt apply, including truncation of real inputs and their finite conversion ranges.
  • Follow the NZ phase protocol.

catNatNZ

(number by3 --) Appends unsigned decimal text with optional groups of three digits without adding the String's trailing zero terminator.
  • The input is converted to Nat64 before formatting. Negative signed inputs therefore use their unsigned representation; real inputs lose their fractional part and must be within the finite conversion range.
  • by3 is a Cond; TRUE inserts comma separators and FALSE disables grouping.
  • Zero produces one digit, 0.
  • Follow the NZ phase protocol.

catUint

(number --) Intended to append unsigned integer text and restore the terminator.

Known issue: catUint calls the unavailable catUintNZ helper; any use fails at compile time with «catUintNZ», Name was not found. Unsigned numbers can be appended through cat.

catHex

(number --) Appends the Nat64 conversion of number in uppercase hexadecimal, without a prefix or leading zero padding, and restores the terminator.
  • Zero produces 0. Negative signed integers use their 64-bit unsigned representation; -1 produces FFFFFFFFFFFFFFFF. Real inputs must be finite and in the Nat64 conversion range; they lose their fractional part. Cond inputs produce 0 or 1.

catHexNZ

(number --) Appends the Nat64 representation of number as uppercase hexadecimal without a prefix, leading zero padding, or the String's trailing zero terminator.
  • The input is cast to Nat64; zero produces 0 and hexadecimal digits from 10 through 15 use uppercase A through F.
  • Follow the NZ phase protocol.

catFloat

(number by3 --) Appends Real32 or Real64 text and restores the terminator.
  • Either zero produces 0.0; positive infinity, negative infinity, and NaN produce inf, -inf, and nan. Other negative values have a minus sign. Finite output retains a decimal point and at least one fractional digit.
  • For Real64, fixed notation is selected for base-10 orders -4 through 8; for Real32, -3 through 7. Other orders use lowercase e followed by a decimal exponent, without a plus sign for a positive exponent.
  • With base-10 order k, fixed notation uses a rounding precision of min(6, 8 - k) decimal places for Real64 or min(5, 7 - k) for Real32. Exponential notation uses six or five decimal places of rounding precision in the significand, respectively. Trailing fractional zeros are omitted except for one required digit. Rounding can leave an unnormalized significand, such as 10.0e9.
  • by3 is a Cond. FALSE disables grouping; TRUE enables comma grouping.

Known issue: decimal scaling can overflow for very small finite values, such as 1e-303 Real64 and 1e-34 Real32. Their formatted output is not reliable.

Known issue: grouping real numbers can add a leading comma and group fractional digits. For example, grouped catMany appends ,123.5 for 123.5 By3 and 0.123,456 for 0.123456 By3; catFloat with by3 TRUE produces the same output. Avoid By3 for real-valued items; call catFloat with FALSE.

catFloatNZ

(number by3 --) Appends Real32 or Real64 formatting without adding the String's trailing zero terminator.
  • Zero, positive infinity, negative infinity, and NaN produce 0.0, inf, -inf, and nan.
  • by3 is a Cond; TRUE enables the same grouping behavior as catFloat, and FALSE disables grouping.
  • The formatting rules and known issues of catFloat apply, including its precision, notation, and tiny-value limits.
  • Follow the NZ phase protocol.

catBy3NZ

(arg --) Appends a supported integer or real value with grouping enabled, without adding the String's trailing zero terminator.
  • Signed integers, unsigned integers, and real values use catIntNZ, catNatNZ, and catFloatNZ, respectively, with grouping enabled.
  • An unsupported object raises object is not supported for string concatenation.
  • Follow the NZ phase protocol.

catCondNZ

(cond --) Appends the textual form of a Cond without adding the String's trailing zero terminator.

cat

(object --) Appends one supported object and restores the terminator.
  • Cond produces TRUE or FALSE. Integer inputs use ungrouped decimal text; real inputs use catFloat formatting without grouping. Char is encoded as UTF-8; Text, StringView, String, and supported byte Spans contribute their bytes. Supported Span inputs are Nat8 Span and Nat8 Cref Span; a Nat8 Array is not accepted directly, pass its span instead.
  • A source that aliases the receiver must be copied first; see the known-issue note at catString.
  • An unsupported object raises object is not supported for string concatenation.

catNZ

(arg --) Appends one supported object without adding the String's trailing zero terminator.
  • Char is UTF-8 encoded; Text, StringView, String, and supported byte spans contribute their bytes.
  • A Cond produces "TRUE" or "FALSE"; integer and real inputs use ungrouped decimal formatting.
  • Supported byte spans have item schema Nat8 and are either Span or Cref Span; a Nat8 Array is not accepted directly.
  • An unsupported object raises object is not supported for string concatenation.
  • Follow the NZ phase protocol.

catMany

(source --) Appends items from a List, Tuple, Index, or Indexable accepted by toIndex, without inserting separators. An unmarked item uses cat. A following Hex selects catHex for the preceding item; a following By3 selects grouped integer or real formatting. The item must be accepted by that formatter, and the marker is consumed.
  • Markers are recognized by the presence of HEX or BY3 fields; HEX takes precedence when both are present.
  • An invalid source raises Built-in tuple, Index, or Indexable expected. An item rejected by cat or by grouped formatting raises object is not supported for string concatenation; this includes a marker processed as an unmarked item.
  • Supply an independent indexed source. Copy any items that alias the receiver before building the sequence; for a String or StringView item, use toString. Do not convert the sequence itself to String. See the known-issue note at catString.

makeNZ

(--) Removes the final backing-array item when present, preparing a String for non-terminated appends.
  • It assumes the removed item is the trailing zero and does not inspect its value.
  • It is a no-op for an empty backing array, and removing the item leaves capacity unchanged.
  • Follow the NZ phase protocol.

makeZ

(--) Appends a zero byte when the backing array is nonempty, restoring the String's trailing zero terminator.
  • It leaves an empty backing array empty; on a nonempty array the appended Nat8 zero is excluded from the logical String.size.
  • Follow the NZ phase protocol.

resize

(size --) Sets the logical byte count to size, a nonnegative Int32.
  • A positive size leaves size + 1 backing bytes, ending in zero. Zero clears the backing item count without releasing reserved storage.
  • Growth does not initialize new content; the former terminator can become a logical zero byte.

catSymbolCode

(codepoint --) Converts codePoint to Int32, then to Char, and appends it. Conversion can truncate a wider input before the code-point check.
  • The resulting code point must be in 0 through 0xD7FF or 0xE000 through 0x10FFFF. Assertion checking reports invalid UTF-8 code point for an invalid value; disabling runtime assertions does not remove this precondition.

getStringView

(-- view) Returns a non-owning StringView over the String's current bytes.
  • The view is empty for an empty String; it does not own or retain the String's buffer. The result permits immutable byte access, even from a mutable receiver.

span

(-- span) Returns a Span over the logical bytes. It carries a mutable Nat8 reference from a mutable String receiver and an immutable reference otherwise.

Examples

"String"  use
"control" use

{} () {} [
  text: "abc" toString;
  text.size printStack _:;
  @text.getStringView.size printStack _:;
] "main" exportFunction

Expected Output During Compilation

Int32
Int32

Formatting Example

"String"  use
"control" use

{} Int32 {} [
  text: String;
  (123456 By3 ", " 255 Hex) @text.catMany
  @text.getStringView print
  0
] "main" exportFunction

Expected Output

123,456, FF

Numeric helper

intPow

(base exponent -- result) Raises a numeric base to an Int32 exponent.
  • The exponent zero returns one in the base's schema, and the result otherwise retains the base's schema.
  • A negative exponent first replaces the base with one divided by the base in that schema; integer bases therefore use integer division, while real bases produce reciprocals.
  • The exponent must be greater than -2147483648.

Known issue: the minimum Int32 exponent, -2147483648, cannot be negated in its schema. With a known exponent the loop fails during compilation; with an unknown exponent it does not terminate; do not pass that exponent.

Text helpers

strlen

(address -- length) Returns the C zero-terminated byte length at address.
  • The result excludes the zero byte and performs no UTF-8 validation. address and length have schema Natx. address must point to readable storage containing a zero terminator.

addTerminator

(source -- terminated) Appends a zero byte to known Text, or to an owned String conversion of a non-Text input.
  • Known Text returns Text with one additional zero byte, even if it already ends in zero.
  • A non-Text input is converted with toString; the appended zero counts toward the result's logical size, before its ordinary trailing terminator.
  • Use toString before adding a terminator when the Text is not known at compile time.

assembleString

(source -- string) Creates an owned String by appending the items in source.
  • Accepts the same indexed sources, items, and formatting markers as String.catMany.

By3

(-- marker) Provides the marker consumed by String.catMany for grouped decimal output.

Hex

(-- marker) Provides the marker consumed by String.catMany for uppercase hexadecimal output.

hash

(text -- hash) Returns the StringView byte hash for Text.

print

(object --) Formats a supported object and writes its bytes to standard output without adding a line feed.
  • Accepts every non-Text input supported by String.cat; a non-String object is converted with toString first. Use control's print for Text. Does not report write errors or retry short writes.

printList

(source --) Assembles source into a String and prints it.
  • Accepts the same indexed sources, items, and formatting markers as String.catMany.

toString

(object -- string) Converts a supported object to an owned String.
  • Unsupported inputs fail with object is not supported for string concatenation.
  • Accepts the same inputs as String.cat, including known and unknown Text. Text, StringView, and String contents are copied into the result; a String input is not moved, even when supplied through a mutable reference.

splitString

(source -- result) Splits Text, StringView, or String into non-owning StringViews, one for each byte range consumed by REPLACE-mode decoding.
  • The ranges retain the original bytes, including malformed bytes; they do not contain substituted replacement characters.
  • success is always TRUE and errorOffset is always -1 in this version; the failure branch is unreachable because REPLACE-mode decoding consumes at least one byte from a nonempty remainder.
  • Empty input produces an empty chars Array. The source storage must outlive the returned views.

Examples

"String"  use
"control" use

{} () {} [
  parts: "你好" splitString;
  parts.success printStack _:;
  parts.chars.size printStack _:;
] "main" exportFunction

Expected Output During Compilation

Cond Cref
Int32

Printing Example

"String"  use
"control" use

{} Int32 {} [
  ("left=" 42) printList
  0
] "main" exportFunction

Expected Output

left=42

UTF-8 helpers

REPLACEMENT_CHARACTER

(-- char) Provides the Unicode replacement character U+FFFD.

isHeadUnit

(unit -- cond) Accepts Nat8. Returns TRUE for an ASCII byte or a byte in the 0xC0 through 0xF7 range.
  • Tests the leading-byte pattern only, not the validity of a UTF-8 sequence.

isValidCodepoint

(codepoint -- cond) Accepts Int32. Returns TRUE for 0 through 0xD7FF or 0xE000 through 0x10FFFF, excluding surrogates.

toChar

(source -- char) Converts an Int32 code point or Text to Char. An Int32 input must be a valid Unicode scalar value. Text is decoded in TRUST mode; only the first result is used, trailing characters are ignored, and empty Text returns REPLACEMENT_CHARACTER.
  • The resulting code point must be in 0 through 0xD7FF or 0xE000 through 0x10FFFF. Assertion checking reports invalid UTF-8 code point for an invalid value; disabling runtime assertions does not remove this precondition.

Utf8DecoderMode

(-- modes) Pushes a namespace object; Utf8DecoderMode.REPORT, Utf8DecoderMode.TRUST, and Utf8DecoderMode.REPLACE yield known Int32 values 0, 1, and 2.

decode

(source -- decoder) Creates a TRUST-mode UTF-8 decoder for Text, TextIter, StringView, or String.

decodeUtf8

(source mode -- decoder) Creates a decoder whose next returns one decoded value and a Cond.
  • Accepts Text, a List or Tuple of Nat8 units, a byte iterator, or an object whose iter method supplies one.
  • mode is a known Int32 selected at compile time.
  • Advance the decoder through a mutable receiver. next has stack effect (-- value cond). REPORT returns Int32 values; TRUST and REPLACE return Char values.
  • At exhaustion, REPORT returns -1 with FALSE; TRUST and REPLACE return REPLACEMENT_CHARACTER with FALSE.
  • REPORT returns -1 with TRUE for a detected malformed sequence; later calls continue at the next unread byte.
  • REPLACE returns REPLACEMENT_CHARACTER with TRUE for a detected malformed sequence. Bytes already read, including an offending continuation byte, are consumed rather than retried; C2 41 42 produces a replacement followed by B.
  • TRUST requires well-formed UTF-8. Assertion checking can report ill-formed UTF-8 sequence; malformed input has no supported behavior with checking disabled.

Known issue: advancing a TRUST decoder over a short source with a known length can fail at compile time with ill-formed UTF-8 sequence even for valid input. This affects Text supplied directly to decodeUtf8 as well as StringView and TextIter. For Text, use decode instead; for a view or iterator, apply dynamic before decoding.

getCodePointAndSize

(data size -- codepoint byteSize) Deprecated helper that decodes the first code point with REPLACE mode and returns its code point and consumed byte count.
  • data is a Nat8 reference and size is a nonnegative Int32 byte count covering readable storage. getCodePointAndSize returns a Nat32 code point followed by an Int32 count; getCodePointSize returns only the count. For getCodePointAndSize, empty input returns code point 65533 and count zero; malformed input returns code point 65533 and the consumed count, not necessarily one. getCodePointSize returns the same count, including zero for empty input.

getCodePointSize

(data size -- byteSize) Deprecated helper that returns the byte count consumed by the first REPLACE-mode decoded code point.
  • Accepts the same data reference and size as getCodePointAndSize. Returns an Int32 consumed-byte count, including zero for empty input.

makeStringIter2

(data size -- iter) Creates an iterator over a byte reference and its UTF-8 byte count.
  • data is an immutable Nat8 reference and size is a nonnegative Int32 byte count. The storage must cover that range and outlive the iterator and returned views. Convert a mutable input view with data const size makeStringIter2.
  • valid is TRUE while the current REPLACE-mode code point has a nonzero byte size, and get returns a non-owning StringView for that byte range.
  • next advances by the current code-point byte size, decreases the remaining count, and computes the next size.

See also