Eskiueskiu v0.7.0

Eskiu Design Decisions

This document records the reasoning behind the architectural and language choices in Eskiu. Each section states the decision, lists the alternatives considered, and explains why they were rejected.


Origin and vision

Eskiu was built in response to a concrete problem: compute-intensive work usually gets split across several languages, each strong at one thing. A typical pipeline uses C for performance-critical code, Go for concurrent services, C++ for libraries, and Python for glue, and every seam between them adds a toolchain, its own idioms, and an interop cost.

Eskiu's answer is a single language with the power of C and the immediacy of a scripting language: it compiles to native code through LLVM, yet eskiuc run file.esk runs a .esk file directly, the way you would a Python or Ruby script. It starts from solid systems foundations and grows upward into the domain, so it can cover that ground without trading native performance for ergonomics.

Foundation phase. Establish a language that can do everything C can do: native performance, explicit memory, direct access to any C library. Validate it against real production code before claiming it works. This phase is complete: a cryptographic pipeline runs entirely in Eskiu, 2.5× faster than the reference C.

v0.1 milestone, shipped. Booting Eskiu code on bare metal in QEMU without libc, the classic proof-of-concept for a systems language. Delivered in v0.1.0 via inline assembly, freestanding mode, and volatile: an ARM64 kernel written in Eskiu boots in QEMU (-M virt) on the PL011 UART, with no libc or C runtime.

Domain specialisation phase. Once the systems foundation is stable, make the domain types that high-throughput services actually work with first-class in the language, without losing general systems capability.

Stage Eskiu Reference C
QR extraction 71.7 ms 185.5 ms
Crypto (AES+RSA) 2.8 ms 2.9 ms
Output decode < 1 ms 0.5 ms
Total 74.4 ms 188.9 ms

2.5× faster than the reference C. The crypto stage matches hand-written C within 0.1 ms.

Every design decision below was evaluated against two questions: does it get the decoder working correctly, and does it serve the long-term domain specialisation goal?


1. C-Style Syntax

Decision: Eskiu uses C-style syntax: braces for blocks, semicolons as statement terminators, type-before-name declarations (int x = 5), for (init; cond; step), explicit return types before function names.

Alternatives considered:

Why C-style was chosen: The decoder target requires calling extern C functions (EVP_DecryptInit_ex, zxing::ImageView), declaring C-compatible structs (uint8[858] fixed arrays, pointer fields), and reasoning about memory layout in terms the target audience already holds. C syntax is the least surprising bridge between "the language I'm writing" and "the C ABI I'm calling into."


2. No Garbage Collector

Decision: Eskiu has no garbage collector. Memory is managed explicitly with alloc<T>(n) (which emits malloc) and free(ptr) from the <mem> stdlib. There is no reference counting and no tracing GC. (As of v0.2.0 the stdlib does ship arena/bump/pool allocators in <alloc> over the alloc_with primitive (see below), but they are opt-in library types, not built into the language runtime.)

Alternatives considered:

Why no GC: The pipeline allocates known-size buffers (alloc<uint8>(858), alloc<uint8>(4096)), uses them, and frees them in a deterministic sequence. Explicit alloc<T>/free matches the mental model exactly and produces the same object code as hand-written C.


3. No Borrow Checker

Decision: Eskiu does not enforce ownership or borrowing rules at compile time. Pointers (*T) are raw; aliasing is permitted; there is no lifetime annotation system.

Alternatives considered:

Why rejected: The decoder code pattern is: allocate a buffer, pass a pointer to a C library function, check the result, free the buffer. This pattern is safe by inspection, and enforcing it mechanically would require the programmer to teach the borrow checker about C library semantics via annotations, which is the same work as writing the C directly. The explicit-control goal means the programmer is trusted to manage pointers correctly, the same assumption made in C.


4. LLVM as Backend

Decision: Eskiu emits LLVM IR via IRBuilder<> and uses TargetMachine + legacy::PassManager to produce native .o files. No custom backend.

Alternatives considered:

Why LLVM: A single LLVM IR module covers arm64 (Apple Silicon), x86-64 (Linux/Windows), RISC-V, and WASM with no changes to the compiler front end. getDefaultTargetTriple() + InitializeNativeTarget() means the same compiler binary produces correct code on the machine it runs on. The emitObjectFile() function is 15 lines; LLVM does the rest.


5. Visitor Pattern for Passes

Decision: Every compiler pass (type checker, code generator, AST printer) implements the ASTVisitor abstract class. Each AST node holds a pure-virtual accept(ASTVisitor*) method. Dispatch is double-dispatch: node->accept(pass) calls pass->visit(node) with the node's concrete type resolved at compile time through the node's own accept override.

Alternatives considered:

Why visitor: Adding a new pass means implementing a new ASTVisitor subclass. Existing AST nodes and existing passes are not touched. The compiler has three pass implementations today (ASTPrinter, TypeChecker, CodeGen); each was added without modifying the other two. The ASTVisitor base class declares one pure-virtual visit() overload per concrete node type (47 today), so the compiler enforces coverage: a new node type added to ast.h that lacks a visit() overload causes a compile-time error in all three passes simultaneously.


6. BlockItem = std::variant<DeclPtr, StmtPtr>

Decision: BlockStmt::items is std::vector<std::variant<DeclPtr, StmtPtr>> rather than two separate lists or a common Stmt base for declarations.

Alternatives considered:

Why variant: std::variant<DeclPtr, StmtPtr> preserves both the ordering (a single vector iterated in sequence) and the semantic distinction (visitors dispatch to the declaration or statement path with std::holds_alternative / std::get). The type checker's visit(BlockStmt*) iterates items in order and calls decl->accept(this) or stmt->accept(this) based on the active alternative. This matches C semantics exactly: a variable declared in the middle of a block is visible to all code after it in the same block, but not before.


7. Monomorphic Templates (Not Generic/Polymorphic)

Decision: Template structs and template functions are instantiated monomorphically: Result<int, string> and Result<bool, string> are two separate LLVM struct types (%Result_int_string and %Result_bool_string), emitted independently. There is no polymorphic representation (no type-erased value + vtable, no boxing).

Alternatives considered:

Why monomorphic: Monomorphization is the simplest implementation that produces correct, fast code. Each instantiation is a concrete LLVM struct type with known field offsets. getTypeFromString("Result<int,string>") calls ensureTemplateInstantiated("Result_int_string", "Result", {"int","string"}), which creates %Result_int_string = type { i32, i32, ptr } the first time it is needed. Subsequent uses find it in structTypes. The cost is code size (multiple copies of a template function body), which is acceptable for the decoder's small number of instantiations.


8. Go-Style Structural Interfaces

Decision: An interface is satisfied implicitly by having all the required methods. No implements keyword, no declaration that a struct satisfies an interface. If struct Point has a method float sum() and interface Summable declares float sum(), then Point satisfies Summable, automatically, structurally.

Alternatives considered:

Why structural: The type checker's structSatisfiesInterface(functionSignatures, structName, iface) function iterates the interface's method signatures and checks whether structName_methodName exists in functionSignatures. This check requires no annotation from the programmer; it falls out of the method registration that already happens in the type checker's first pass. Structural interfaces also let future stdlib code define interfaces that existing structs satisfy without modifying those structs.


9. Result<T,E> Before Exceptions

Decision: Error propagation in Eskiu is done via Result<T,E> values (struct Result<T,E> { int ok; T value; E error; }). This was implemented before exceptions, and the decoder was written against it (checking result.ok and branching manually). Both now ship in v0.1.0: exceptions (try/catch/finally/throw) landed as well, and the postfix ? operator was added to make Result-based code ergonomic: divide(a, b)? returns the Err early or unwraps the value, removing the manual if (!r.ok) return r; boilerplate. The design rationale below (why Result<T,E> came first) still holds.

Alternatives considered:

Why Result<T,E> first: Result<T,E> is zero-cost when ok == 1: the caller accesses result.value directly via GEP without any dynamic dispatch or heap allocation. The struct layout is known at compile time. Error propagation is explicit in the code (the ? operator is still a visible, checkable control-flow point, not a hidden unwind), which matches the explicit-control philosophy of the language. Exceptions were added alongside it without removing Result<T,E>; they serve different use cases (recoverable domain errors vs. unrecoverable runtime failures).


10. Hand-Written Recursive-Descent Parser

Decision: The parser is a hand-written recursive-descent parser with no parser generator. The grammar is encoded as a call graph: parseProgram calls parseDeclaration, which calls parseFunctionDecl, parseStructDecl, etc.

Alternatives considered:

Why recursive descent: Error messages are the primary reason. When consume(SEMICOLON, "expected ';'") fails, the throw site is the exact function that expected the semicolon, one function call away from the code the user wrote. The parser can provide "expected ';' after variable declaration" rather than "syntax error." Backtracking is controlled and explicit: three call sites use size_t savePos = current and reset on failure. The parser also handles the template-call ambiguity (ident<T>(args) vs ident < T) with speculative parsing that resets current on failure; this pattern is awkward in a grammar table but natural in recursive descent.


11. For-Loop Declaration Scope Fix

Decision: The init clause of a for loop can be a variable declaration (for (int i = 0; i < n; i++)), and that variable is scoped to the entire loop (condition, step, and body), not just the init clause.

Problem: The parser wraps a for-loop init declaration in a BlockStmt (a single-item block containing the DeclPtr). If the type checker treats this BlockStmt as an ordinary block, it pushes a scope for the init, evaluates the declaration, then pops the scope before checking the condition, making i invisible in i < n.

How it was fixed: visit(ForStmt*) in the type checker detects when init is a BlockStmt and iterates its items directly in the for-loop's own scope rather than delegating to visit(BlockStmt*). The scope push/pop wraps the entire for-loop body:

pushScope()
  process init BlockStmt items (defines i)
  visit condition (i < n, sees i)
  visit body (sees i)
  visit step (i++, sees i)
popScope()

Alternatives considered:

Why the fix matters: The decoder code uses for (int i = 0; i < len; i++) throughout. Without the scope fix, every for-loop with a declaration init fails at the type checker with "undefined variable i."


12. Lazy Template Instantiation

Decision: Template structs and functions are not instantiated when they are declared. They are instantiated the first time they are used. struct Result<T,E> in the source file adds an entry to templateDecls["Result"] but emits no LLVM IR. The first let r: Result<int, string> triggers ensureTemplateInstantiated("Result_int_string", "Result", {"int","string"}) which creates %Result_int_string = type { i32, i32, ptr }.

Alternatives considered:

Why lazy: Lazy instantiation means the template machinery is activated only when a concrete type is needed. The type checker's normalizeType("Result<int,string>") performs substitution and registers StructInfo for "Result_int_string" the first time the type appears. Codegen's ensureTemplateInstantiated is idempotent: it checks structTypes.count(mangled) before doing any work. Both the type checker and codegen interpret the type spelling through the same ty::Type::parse grammar (see Decision 14), so the substitution and instantiation logic is shared rather than reimplemented per phase. Template functions follow the same pattern: visit(TemplateCallExpr*) mangles the name, saves/restores the insert point, sets typeParamOverride, and re-visits the FunctionDecl body exactly once per unique type argument combination.


13. Integer Width Coercion in Comparisons

Decision: When a comparison (==, !=, <, etc.) is emitted for operands of different integer widths, the narrower operand is zero-extended (ZExt) or sign-extended (SExt) to match the wider type before the LLVM ICmp instruction is emitted.

Problem: LLVM's CreateICmpEQ(left, right) requires both operands to have the same type. The decoder computes with uint8 values (from a uint8[858] buffer) and compares them to int constants. Without coercion, ICmpEQ(i8 %byte, i32 0x30) causes an LLVM assertion failure at IR verification time.

Implementation: visit(BinaryExpr*) checks whether the comparison involves integer operands of different widths. If left is i8 and right is i32, it calls CreateZExt(left, i32) before emitting ICmpEQ. Zero-extension is used (not sign-extension) because the values being compared are byte values from a buffer, treated as unsigned. The same coercion is applied in visit(VarDecl*) for assignments and in emitStructInitInto() for struct field initialization.

Alternatives considered:

Why ZExt: Zero-extension from i8 to i32 correctly represents unsigned byte values in the range 0–255 as i32 values in the range 0–255, preserving the ordering relationship for all relational operators.


14. Structured ty::Type IR and a Single Type Resolver

Decision: Types are represented internally by a structured ty::Type IR (sema/type.{h,cpp}) with one grammar interpreter, ty::Type::parse, and the type checker is the single resolver: it resolves every expression's type into a per-expression table that codegen consumes. Codegen does not re-derive expression types; its getTypeFromString dispatches on ty::Type::parse, the same grammar interpreter the type checker uses.

Alternatives considered:

Why a structured IR + one resolver: A ty::Type value carries the structure (nominal name, type arguments, pointer levels) directly, with parse / str / substitute / nominalName as the operations every phase needs, so substitution and nominal-name extraction happen once, in one place, instead of being re-derived by each ad-hoc string strip. Making the type checker the sole resolver and having codegen consume its table means there is exactly one answer for any expression's type. ty::Type::parse is the single grammar interpreter both phases share, which structurally closes the two-evaluator divergence risk. The migration was behavior-preserving (gated by the golden-IR oracle) and, once the single resolver landed, fixed the three latent miscompiles above.


15. Opt-In Memory Safety (Zig-Flavored)

Decision: Make the language safer with compile-time checks and opt-in runtime guards, on the Zig side of the line, not the Rust side. Bare *T stays C-nullable and existing code compiles unchanged; the new safety surface is reached for deliberately. The batch: defer / errdefer for cleanup that runs on every exit path (a codegen cleanup-stack, LIFO); the slice type T[], a fat pointer that carries its length so bounds are knowable; the must_use qualifier that rejects a discarded result (the stdlib's alloc is marked, so a forgotten allocation is a compile error); --safe, an off-by-default build mode that bounds-checks array and slice indexing and traps on a violation; and the checked nullable pointer ?*T, which cannot be dereferenced until proven non-null (if (x != null) narrows it) and lowers to a bare pointer, so the safety is entirely at compile time.

Alternatives considered:

Why opt-in compile-time safety: It composes with the existing model instead of replacing it. defer / errdefer / must_use / ?*T are all resolved at compile time with no runtime cost, and --safe is the one runtime guard, off by default. This continues the v0.4 correctness arc (tightening the type system) rather than introducing a new ownership discipline, so a C programmer can pick up each feature independently without relearning how to hold a pointer.


Summary

The decisions above form a consistent position: Eskiu is an explicit-control systems language that targets programmers who already think in C, need to call C libraries, and want near-C performance without writing C. The compiler is designed to be read and modified by one to three people, so implementation simplicity (hand-written parser, recursive-descent, visitor pattern, no parser generator) is a real constraint. LLVM handles the hard parts of code generation; the front end's job is to produce correct IR quickly.

The INE decoder benchmark is the north star. Every decision that made the benchmark harder to hit was rejected; every decision that kept the code close to C semantics while adding safety margins (structural interfaces, Result<T,E>, two-pass type checking) was accepted.