Parser State

Types and utilities for managing parser state, including position tracking and state manipulation.

Overview

The state module provides types and utilities for managing parser state throughout the parsing process. It includes types for representing source positions, parser state, and parser output, as well as utility functions for state manipulation.

import { State, ParserState, ParserOutput, SourcePosition } from 'parserator';

Types

ParserState

Represents the complete state of a parser at any point during parsing. Contains the input being parsed, current position, and optional debugging/context information.

type ParserState = {
  source: string;
  offset: number;
  line: number;
  column: number;
  debug?: boolean;
  labelStack?: string[];
  committed?: boolean;
}

source · string · required

The complete original input string

offset · number · required

Current byte offset from start of input (0-indexed)

line · number · required

Current line number (1-indexed)

column · number · required

Current column number (1-indexed)

debug · boolean

Whether debug mode is enabled for detailed error reporting

labelStack · string[]

Stack of parsing context labels for error reporting

committed · boolean

Whether the parser has committed to this parse path

const state: ParserState = {
  source: "hello world",
  offset: 0,
  line: 1,
  column: 1,
  debug: true,
  labelStack: ["expression", "identifier"],
  committed: false
};

ParserOutput

Represents the output of a parser operation, containing both the updated state and the parsing result (either success or error).

type ParserOutput<T> = {
  state: ParserState;
  result: Either<T, ParseErrorBundle>;
}

state · ParserState · required

The parser state after the operation

result · Either<T, ParseErrorBundle> · required

Either a successful result of type T or a ParseErrorBundle

const output: ParserOutput<string> = {
  state: newState,
  result: Either.right("parsed value")
};

ParserOutput Factory

Factory function for creating ParserOutput objects.

const ParserOutput = <T>(
  state: ParserState,
  result: Either<T, ParseErrorBundle>
): ParserOutput<T>

state · ParserState · required

The parser state after the operation

result · Either<T, ParseErrorBundle> · required

Either a successful result or error bundle

import { Either } from 'parserator';

const successOutput = ParserOutput(newState, Either.right("success"));
const errorOutput = ParserOutput(oldState, Either.left(errorBundle));

SourcePosition

Represents a position within source code with line, column, and byte offset. All values are 1-indexed for human readability.

type SourcePosition = {
  line: number;
  column: number;
  offset: number;
}

line · number · required

Line number (1-indexed)

column · number · required

Column number (1-indexed)

offset · number · required

Byte offset from start of input (0-indexed)

const position: SourcePosition = {
  line: 3,      // Third line
  column: 15,   // 15th character on that line
  offset: 42    // 42nd character from start of input
};

Spanned

A tuple containing a parsed value and its span information.

type Spanned<T> = [value: T, span: Span]

State Utilities

The State object provides static methods for creating and manipulating parser state.

State.fromInput

Creates a new parser state from an input string.

State.fromInput(input: string): ParserState

input · string · required

The input string to parse

state · ParserState

A new parser state initialized at the start of the input

const state = State.fromInput("hello world");
// {
//   source: "hello world",
//   offset: 0,
//   line: 1,
//   column: 1
// }

State.remaining

Gets the remaining unparsed portion of the input.

This allocates a new string. Prefer peek() or charAt() for better performance.

State.remaining(state: ParserState): string

state · ParserState · required

The current parser state

remaining · string

The remaining input string from current offset

const state = State.fromInput("hello world");
const consumed = State.consume(state, 6);
State.remaining(consumed); // "world"

State.charAt

Gets the character at the current offset without allocating.

State.charAt(state: ParserState): string

state · ParserState · required

The current parser state

char · string

The character at current offset, or empty string if at end

const state = State.fromInput("hello");
State.charAt(state); // "h"

State.startsWith

Checks if remaining input starts with the given string, without allocating.

State.startsWith(state: ParserState, str: string): boolean

state · ParserState · required

The current parser state

str · string · required

The string to check for

result · boolean

True if remaining input starts with str

const state = State.fromInput("hello world");
State.startsWith(state, "hello"); // true
State.startsWith(state, "world"); // false

State.consume

Creates a new state by consuming n characters from the current state.

State.consume(state: ParserState, n: number): ParserState

state · ParserState · required

The current parser state

n · number · required

Number of characters to consume

newState · ParserState

A new state with n characters consumed and position updated

const state = State.fromInput("hello");
const newState = State.consume(state, 2);
// newState.offset === 2
// State.remaining(newState) === "llo"

Throws an error if attempting to consume more characters than remaining.

State.consumeString

Creates a new state by consuming a specific string from the current state.

State.consumeString(state: ParserState, str: string): ParserState

state · ParserState · required

The current parser state

str · string · required

The string to consume

newState · ParserState

A new state with the string consumed and position updated

const state = State.fromInput("hello world");
const newState = State.consumeString(state, "hello");
// State.remaining(newState) === " world"

Throws an error if the input doesn’t start with the specified string.

State.consumeWhile

Creates a new state by consuming characters while a predicate is true.

State.consumeWhile(
  state: ParserState,
  predicate: (char: string) => boolean
): ParserState

state · ParserState · required

The current parser state

predicate · (char: string) => boolean · required

Function that tests each character

newState · ParserState

A new state with matching characters consumed

const state = State.fromInput("123abc");
const newState = State.consumeWhile(state, c => /[0-9]/.test(c));
// State.remaining(newState) === "abc"

State.peek

Gets the next n characters from the input without consuming them.

State.peek(state: ParserState, n: number = 1): string

state · ParserState · required

The current parser state

n · number

Number of characters to peek (default: 1)

chars · string

The next n characters as a string

const state = State.fromInput("hello");
State.peek(state, 3); // "hel"
State.peek(state);     // "h"

State.isAtEnd

Checks if the parser has reached the end of input.

State.isAtEnd(state: ParserState): boolean

state · ParserState · required

The current parser state

atEnd · boolean

True if at end of input, false otherwise

const state = State.fromInput("hi");
State.isAtEnd(state); // false
const endState = State.consume(state, 2);
State.isAtEnd(endState); // true

State.move

Creates a new state by moving to a specific offset position in the source.

State.move(state: ParserState, moveBy: number): ParserState

state · ParserState · required

The current parser state

moveBy · number · required

Number of characters to move forward from current position

newState · ParserState

A new state at the target position

State.computePosition

Computes the actual line and column for a given offset.

This is expensive (O(offset)) so only call when creating errors.

State.computePosition(state: ParserState): ParserState

state · ParserState · required

The current parser state

updatedState · ParserState

Updated state with correct line/column

State.printPosition

Creates a human-readable string representation of the current parser position.

State.printPosition(state: ParserState): string

state · ParserState · required

The current parser state

position · string

A formatted string showing line, column, and offset

const state = { source: "hello", offset: 5, line: 1, column: 6 };
const posStr = State.printPosition(state);
// Returns: "line 1, column 6, offset 5"

State.toPosition

Creates a SourcePosition from the current parser state.

State.toPosition(state: ParserState): SourcePosition

state · ParserState · required

The current parser state

position · SourcePosition

A SourcePosition object

const state = State.fromInput("hello");
const position = State.toPosition(state);
// { line: 1, column: 1, offset: 0 }

How State Flows Through Parsing

Parser state flows through the parsing process in the following way:

1. Initial State: Created from input using State.fromInput(input)
2. Parser Execution: Each parser receives the current state and returns a new state
3. State Updates: Successful parsers advance the offset, failed parsers may preserve the original state
4. Sequencing: State flows from one parser to the next in sequential operations
5. Backtracking: Failed parsers in choice operations reset to the original state
6. Commit Points: Once committed, backtracking is prevented

// Example: State flow through parsing
const initialState = State.fromInput("hello world");

// First parser consumes "hello"
const { state: state1, result: result1 } = stringParser.run(initialState);
// state1.offset = 5

// Second parser consumes " "
const { state: state2, result: result2 } = whitespaceParser.run(state1);
// state2.offset = 6

// Third parser consumes "world"
const { state: state3, result: result3 } = wordParser.run(state2);
// state3.offset = 11