Skip to content
Learn Ghostty

Chapter 07: Incremental VT parser

Let’s teach the program what those bytes mean

Section titled “Let’s teach the program what those bytes mean”

Termio can move bytes, but right now 1B 5B 33 32 6D is just five numbers. In this chapter a Parser turns the stream into small instructions such as “print this byte,” “move for CR,” or “set green style.”

VISIBLE RESULTOne action stream independent of PTY read boundaries
chapter-07-parser
$ zig build run
[entry] ghostty
[main] process started
[app] created
[runtime] initialized
[surface 1] created
[runtime] tick
[pipe child stdout]
child: hello
stdin_tty=no
stdout_tty=no
stderr_tty=no
[pipe child] exited 0
[termio] started
[termio read ready] 72 65 61 64 79 3E
[termio write] 68 65 6C 6C 6F 0A
[termio read reply] 68 65 6C 6C 6F 0D 0A 1B 5B 33 32 6D 72 65 70 6C 79 3A 68 65 6C 6C 6F 1B 5B 30 6D 0D 0A
[parser print] ready>hello
[parser execute] CR
[parser execute] LF
[parser sgr] 32
[parser print] reply:hello
[parser sgr] 0
[parser execute] CR
[parser execute] LF
[termio] child exited 0
[termio] stopped
[surface 1] destroyed
[runtime] terminated
[app] destroyed
[main] process exiting

The live child sequence is read in buffers no larger than three bytes. Parser actions still reconstruct print runs, CR/LF controls, green SGR 32, and reset SGR 0.

Why can’t we just search for escape strings?

Section titled “Why can’t we just search for escape strings?”

Chapter 06 searched for the literal readiness marker ready>. VT syntax is stateful:

ESC arrives now
[ arrives in the next read
32 arrives later
m finally dispatches the action
STATE MACHINEState survives when an escape sequence is split anywhere
groundprint / execute
ESC
escapewait for introducer
[
CSIcollect digits
m
groundemit SGR
… 1B5B 3332 6D …three independent reads → one SGR 32 action

The parser consumes a byte stream, not messages. Ending a call in escape or CSI is normal; the next call resumes that state.

A read can stop anywhere, even in the middle of ESC [ 32 m. That is normal. The Parser must remember where it stopped and continue when the next chunk arrives.

DEPENDENCY FRONTIER

Build only what the current result needs

  1. 00Process entryReconstructed
  2. 01App lifecycleReconstructed
  3. 02Runtime + SurfaceReconstructed
  4. 03Child process + PTYReconstructed
  5. 04I/O + parserYou are here
  6. 05Terminal stateNext limitation
  7. 06Window + GPU + fontsNot introduced

Real Ghostty also started with a small parser

Section titled “Real Ghostty also started with a small parser”

Ghostty did not begin with today’s complete parser. On April 18, 2022 it first encoded the VT state table, then implemented only print and execute. CSI dispatch followed on May 8; OSC, DCS, UTF-8, colon parameters, and many correctness fixes arrived later.

SOURCE ARCHAEOLOGYThen, reconstruction, and now solve different-sized problems
  1. 01
    THEN · 2022-04-18Print and execute first

    The initial state machine emitted printable bytes and C0 controls. CSI dispatch had not been implemented.

    20f9ad3Open first behavior
  2. 02
    RECONSTRUCTION · CHAPTER 07Add one earned CSI family

    Ground, escape, and CSI states emit ASCII print, C0 execute, and the SGR parameters produced by the real child probe.

    chapter-07-parser
  3. 03
    NOW · PINNED MAINCover the VT protocol surface

    Current parser handles CSI, ESC, OSC, DCS, APC, UTF-8, intermediates, separators, passthrough, cancellation, and bounded storage.

    6ad1fe7Open current parser

Our added SGR support is justified by the actual ESC [ 32 m and ESC [ 0 m bytes from Chapter 06—not by a desire to enumerate VT features.

Start with the Parser’s output: what can it tell us?

Section titled “Start with the Parser’s output: what can it tell us?”
ZIG SYNTAX BRIDGE

Represent parser state and emitted actions

Start with the C model, then inspect each highlighted Zig line.

Open in the cumulative Zig/C reference
C MENTAL MODEL

enum state

A closed enum gives each parser mode a named discriminant instead of magic integers.

Zig difference: The compiler checks switch exhaustiveness when every State case is listed.

MEMORY / LIFETIME FLOW
  1. Parser valuecurrent State + partial parameter
  2. next callone input byte
  3. callernull or one Action value

Text version: State is a closed enum. Action is a tagged union whose active tag determines the valid payload. ?Action explicitly represents no emitted action. next mutates persistent parser state and uses a switch expression to return the action produced by one byte.

adaptedsrc/terminal/Parser.zig

Ground/escape/CSI subset emitting ASCII print, C0 execute, and one SGR parameter; full VT and Unicode behavior remain deferred.

const Parser = @This();

const std = @import("std");

pub const State = enum { ground, escape, csi };

pub const Action = union(enum) {
    print: u8,
    execute: u8,
    sgr: u16,
};

state: State = .ground,
parameter: u16 = 0,
has_parameter: bool = false,
actions: [256]Action = undefined,
action_count: usize = 0,

pub fn feed(self: *Parser, bytes: []const u8) !void {
    for (bytes) |byte| if (self.next(byte)) |action| {
        if (self.action_count == self.actions.len) return error.TooManyActions;
        self.actions[self.action_count] = action;
        self.action_count += 1;
    };
}

pub fn next(self: *Parser, byte: u8) ?Action {
    return switch (self.state) {
        .ground => switch (byte) {
            0x1B => state: {
                self.state = .escape;
                break :state null;
            },
            0x00...0x1A, 0x1C...0x1F, 0x7F => .{ .execute = byte },
            else => .{ .print = byte },
        },
        .escape => if (byte == '[') state: {
            self.state = .csi;
            self.parameter = 0;
            self.has_parameter = false;
            break :state null;
        } else state: {
            self.state = .ground;
            break :state null;
        },
        .csi => switch (byte) {
            '0'...'9' => state: {
                self.parameter = self.parameter * 10 + (byte - '0');
                self.has_parameter = true;
                break :state null;
            },
            'm' => dispatch: {
                const value = if (self.has_parameter) self.parameter else 0;
                self.state = .ground;
                break :dispatch .{ .sgr = value };
            },
            else => state: {
                self.state = .ground;
                break :state null;
            },
        },
    };
}

pub fn traceActions(self: *const Parser) void {
    var index: usize = 0;
    while (index < self.action_count) {
        switch (self.actions[index]) {
            .print => {
                std.debug.print("[parser print] ", .{});
                while (index < self.action_count) : (index += 1) switch (self.actions[index]) {
                    .print => |byte| std.debug.print("{c}", .{byte}),
                    else => break,
                };
                std.debug.print("\n", .{});
            },
            .execute => |byte| {
                std.debug.print("[parser execute] {s}\n", .{controlName(byte)});
                index += 1;
            },
            .sgr => |value| {
                std.debug.print("[parser sgr] {d}\n", .{value});
                index += 1;
            },
        }
    }
}

fn controlName(byte: u8) []const u8 {
    return switch (byte) {
        '\r' => "CR",
        '\n' => "LF",
        else => "C0",
    };
}

test "actions do not depend on input chunks" {
    const input = "A\x1b[32mB\x1b[0m\r\n";

    var whole: Parser = .{};
    try whole.feed(input);

    var split: Parser = .{};
    for (input) |byte| try split.feed(&.{byte});

    try std.testing.expectEqual(whole.state, split.state);
    try std.testing.expectEqualSlices(
        Action,
        whole.actions[0..whole.action_count],
        split.actions[0..split.action_count],
    );
}

The tagged union is the Parser’s little vocabulary:

pub const Action = union(enum) {
    print: u8,
    execute: u8,
    sgr: u16,
};

Notice what Parser does not do: it does not move a cursor or color a cell. Like a lexer in a C compiler, it recognizes structure and reports it to the next layer.

Now follow one byte through the state machine

Section titled “Now follow one byte through the state machine”

We only need three states for the bytes we have actually seen:

  • ground: printable bytes and C0 controls;
  • escape: saw ESC and waits for an introducer;
  • csi: collects digits until final byte m.

On m, the parser emits sgr(parameter) and returns to ground. An empty parameter means reset 0.

Put one Parser inside Termio

Section titled “Put one Parser inside Termio”

Termio owns one Parser value and calls parser.feed(bytes) inside its read append path. The parser therefore sees the exact bytes in their actual chunking while preserving state across reads.

adaptedsrc/termio/Termio.zig

Adds persistent parser stream state to the Chapter 06 PTY owner; terminal mutation remains absent.

const Termio = @This();

const std = @import("std");
const Command = @import("../Command.zig");
const ptypkg = @import("../pty.zig");
const Pty = ptypkg.Pty;
const Parser = @import("../terminal/main.zig").Parser;

pty: Pty,
child: Command.Child,
phase_bytes: [4096]u8 = undefined,
phase_len: usize = 0,
parser: Parser = .{},

pub fn init(command: Command, size: ptypkg.winsize) !Termio {
    var pty = try Pty.open(size);
    errdefer pty.deinit();
    const child = try command.startPty(&pty);
    std.debug.print("[termio] started\n", .{});
    return .{ .pty = pty, .child = child, .parser = .{} };
}

pub fn deinit(self: *Termio) void {
    self.pty.deinit();
    if (!self.child.waited) _ = self.child.wait() catch {};
    std.debug.print("[termio] stopped\n", .{});
}

pub fn writeAll(self: *Termio, bytes: []const u8) !void {
    try self.pty.writeAll(bytes);
    traceHex("write", bytes);
}

pub fn readUntil(self: *Termio, marker: []const u8) ![]const u8 {
    self.phase_len = 0;
    var chunk: [3]u8 = undefined;
    while (std.mem.indexOf(u8, self.phase_bytes[0..self.phase_len], marker) == null) {
        const count = try self.pty.read(&chunk);
        if (count == 0) return error.EndOfStream;
        try self.append(chunk[0..count]);
    }
    return self.phase_bytes[0..self.phase_len];
}

pub fn readToEnd(self: *Termio) ![]const u8 {
    self.phase_len = 0;
    var chunk: [3]u8 = undefined;
    while (true) {
        const count = try self.pty.read(&chunk);
        if (count == 0) break;
        try self.append(chunk[0..count]);
    }
    return self.phase_bytes[0..self.phase_len];
}

pub fn wait(self: *Termio) !u8 {
    return self.child.wait();
}

pub fn traceParser(self: *const Termio) void {
    self.parser.traceActions();
}

fn append(self: *Termio, bytes: []const u8) !void {
    if (self.phase_len + bytes.len > self.phase_bytes.len) return error.OutputTooLong;
    try self.parser.feed(bytes);
    @memcpy(self.phase_bytes[self.phase_len..][0..bytes.len], bytes);
    self.phase_len += bytes.len;
}

pub fn traceHex(label: []const u8, bytes: []const u8) void {
    std.debug.print("[termio {s}]", .{label});
    for (bytes) |byte| std.debug.print(" {X:0>2}", .{byte});
    std.debug.print("\n", .{});
}

test "Termio owns interactive PTY reads and writes" {
    const script: []const u8 = "printf 'ready>'; IFS= read -r line; printf 'reply:%s\\n' \"$line\"";
    const command: Command = .{ .argv = &.{ "/bin/sh", "-c", script } };
    var termio = try Termio.init(command, .{});
    defer termio.deinit();
    _ = try termio.readUntil("ready>");
    try termio.writeAll("test\n");
    _ = try termio.readToEnd();
    try std.testing.expectEqual(@as(u8, 0), try termio.wait());
}

The three-byte buffers split the green sequence:

... 1B | 5B 33 | 32 6D ...

Yet only one [parser sgr] 32 appears.

Test the bug we are most likely to write

Section titled “Test the bug we are most likely to write”

The parser test feeds:

A ESC [ 3 2 m B ESC [ 0 m CR LF

once as a whole slice and again one byte per call. It compares final state and every action exactly.

If both inputs produce the same actions, we know our Parser is reading a stream rather than depending on lucky kernel chunk sizes.

Section titled “Print the actions so we can inspect them”

Consecutive print actions are grouped for readability. Controls and SGR remain explicit:

[parser print] reply:hello
[parser sgr] 0
[parser execute] CR
[parser execute] LF

This trace is not terminal state. It is the ordered instruction stream terminal state will consume.

Fidelity

Section titled “Fidelity”
File Status Deferred production behavior
src/terminal/Parser.zig adapted UTF-8, full CSI, ESC dispatch, OSC, DCS, APC, intermediates, cancellation, bounded parameter lists
src/terminal/main.zig adapted exports only Parser
src/termio/Termio.zig adapted owns Parser but no Terminal or renderer wakeup
src/Surface.zig adapted traces actions instead of displaying state

Why is production Ghostty’s parser much larger?

Section titled “Why is production Ghostty’s parser much larger?”
</>
Current parser vocabularysrc/terminal/Parser.zig:1–86Read State, TransitionAction, and Action. Ignore detailed structures for protocol families not introduced here.
Read excerptGitHub
</>
Current transition tablesrc/terminal/parse_table.zig:1–40See state transitions represented as data. Do not attempt to memorize the full table.
Read excerptGitHub

The reconstruction uses a direct switch because three states are still readable. A generated/table-driven representation becomes useful when the protocol surface makes direct branching hard to audit.

Checkpoint

Section titled “Checkpoint”
commit  b4549bf84e5c36d3c760f3e14e0de244f3489e35
tag     chapter-07-parser

We understand actions—where do they live?

Section titled “We understand actions—where do they live?”

Actions disappear after tracing. Chapter 08 introduces durable terminal state:

parser actions → cells + cursor + current style → deterministic debug frame

No renderer or window is needed to prove the state transition.