Jennifer - Milestones
Development is split into milestones. Each milestone produces a working interpreter that runs a strictly larger subset of the language.
M1 - End-to-end MVP
Status: done.
Smallest vertical slice that proves the pipeline (source → tokens → preprocessed tokens → AST → result):
- Types:
int,string def x as int init 5;,$varreferences, method defs (zero-arg, top level),import "file.j";,use io;, single-argprintf- Arithmetic
+ - * / %on ints; comments#and/* */ - Source-context caret in error messages
- Golden-file integration test and TinyGo build verified
Exit criterion: ./jennifer run examples/hello.j prints 42.
M2 - Types, constants, scoping, control flow
Status: done.
Rounds out the “ordinary” feature set:
- New types
float,null,boolwith literals3.14,null,true,false - Uninitialized
def x as T;givesT’s zero value def const NAME as TYPE init VALUE;(reassignment is an error)- Nested block scoping; inner scopes cannot redeclare visible names
- Assignment statement
$x = EXPR; - Comparison
< > <= >= ==,+for string concat,int↔floatpromotion - Escape parsing in
'...'strings (previously only"...") - Control flow:
if/elseif/else,while,for, all requiringboolconditions (no implicit truthiness)
M3 - Methods with parameters and return values
Status: done.
func name(a as int, b as string) { ... }with typed parameters, by-value argument passing, call-site arity + type checksreturn;andreturn EXPR;; recursion workssprintfand format verbs%d %f %s %t %v %%for bothprintfandsprintf- The omnibus
stdlibretired in favor of topic-based libraries;iois the first.
M4 - Polish & ergonomics
Status: done.
- Logical operators
and,or,not(word-based, short-circuit) - Unary minus
- Python-3 division:
/always returnsfloat; newdivkeyword for floor division (//is taken by line comments) - Floats always display with a decimal (
5.0, not5) so the type stays visible - New libraries (all
use-gated):convert,math,strings - Interpreter gained
RegisterConstso libraries can expose constants (PI,E).
M5 - Interpreter improvements
Status: done.
- Cross-file error sources - errors raised inside an imported
.jdisplay the line from the imported file. See technical/interpreter.md > Errors and positions. - REPL -
jennifer repl, persistent globals/methods/imports across inputs, multi-line input via brace balancing, expression results printed. See technical/cli_repl.md > REPL. - REPL line editor - cursor keys, Home/End, word motions, Ctrl+W / Ctrl+U / Ctrl+K, in-memory history (Up/Down), Ctrl+C cancel. Non-TTY stdin falls back to plain line reading. See technical/cli_repl.md > Line editor.
- Auto-loaded
corelibrary - new library kind, pre-imported at startup; writinguse core;is a runtime error. Contents:JENNIFER_VERSION(agit describe-derived build-version constant) andlen(polymorphic over strings now; lists/maps in M6).lenmoved here fromstrings. (M15.4 later promotedlento a language built-in and deletedcore; see M15.4 for the migration.) Version injection details at technical/cli.md. - Formatter -
jennifer fmtre-emits canonical source per user-guide/style-guide.md. Token-level walker so file imports and user-written parentheses survive. See technical/cli_fmt.md > Formatter. - Inspection subcommands -
jennifer tokens <file>dumps the lexer output;jennifer ast <file>dumps the preprocessed AST as JSON. See technical/cli_inspect.md > Inspection. - Underscore-in-constants - constant names became
[A-Z]+(_[A-Z]+)*, enablingMAX_RETRIESand theJENNIFER_VERSIONrename. See technical/lexer.md > Identifier rule. - Documentation overhaul -
docs/technical.mdsplit intodocs/technical/<topic>.md;docs/lib_*.mdmoved todocs/libraries/; newdocs/user-guide/style-guide.md.
M6 - Lists and maps
Status: done.
Two new compound types - list and map - plus the strings library
functions deferred until compound types existed.
- Syntax:
def xs as list of int init [1, 2, 3];,def m as map of string to int init {"a": 1};. Index read/write$xs[i],$m["k"], chains$g[i][j]. Iteration viafor (def x in $coll) { ... }(new keywordin). New tokens[ ] :and keywordslist,map,of,to,in. - Semantics: value-typed (copy on assignment and on
function-parameter binding; no aliasing);
constis deep ($NUMS[0] = ...is a runtime error ifNUMSisconst); out-of-bounds list reads/writes and missing map keys are positioned runtime errors; map iteration is insertion-order deterministic. - Type system:
parser.Typebecame a recursive struct (Element,KeyType,ValType*Typeslots), so nesting likelist of list of intfalls out without depth cap. 3+ levels is a documented code smell. - Stdlib:
core.lenextended to lists and maps;core.has(m, key)for membership tests;strings.split,strings.chars,strings.joinfinished. - Tooling: formatter handles
[...]/{...}per user-guide/style-guide.md (no inner padding, space after,/:, block-vs-map disambiguation via a small brace stack); AST JSON emitter handlesListLit,MapLit,IndexExpr,IndexAssignStmt,ForEachStmt.
See user-guide/types-and-values.md > Lists and maps for the user-facing tour, and technical/grammar.md / technical/interpreter.md for the implementation contract.
M7 - printf modifiers, stdin input, comment/division swap
Status: done.
A breaking syntax change to free up // for integer division and to
allow shebangs, the long-promised format-verb modifier system, and
the first stdin-reading builtins.
- Comments and integer division (BREAKING). Line comments
moved from
//to#, freeing//for floor division (Python 3 shape).divkeyword removed. A Jennifer file can now begin with#!/usr/bin/env -S jennifer run. (s)printfformat-verb modifiers. Each format verb except%vaccepts a pipe-separated, order-independent flag list:%verb[|key=value]*. Modifiers shape presentation only - data transformations (case=upperon strings, markdown rendering, etc.) are explicitly out of scope; libraries do that work. Verbs gained:pad/max/align/mode(%s);pad/fill/align/base/sign/group/sep(%d);prec/trim/sci/pad/align/sign(%f);case(%t); sharednull=empty|null|literal(...)across all four typed verbs.%vdeliberately takes none.- Format-string breaking change.
|immediately after a verb now starts a modifier list. Pre-M7 strings with|as a literal separator ("%d|%d") need either a different separator or the||escape (parallels%%). iostdin input. New builtinsreadLine(),readLine(prompt),eof()- one-line-at-a-time reads with an explicit EOF predicate (while (not eof()) { ... }). Refuses inside the REPL since the line editor owns stdin.- Internals. Builtin signature changed from
func(out io.Writer, args)tofunc(ctx BuiltinCtx, args)so stdin and the REPL flag are plumbed symmetrically with stdout. Mechanical refactor across the ~30 existing builtins.
See:
- libraries/io.md - full modifier and input reference.
- technical/lexer.md and technical/grammar.md - the comment / division syntax change.
- technical/rejected.md - what the modifier
system deliberately doesn’t do (data transformations,
%aaggregate,null=sql/null=skip) and why the literal-pipe lookahead alternative was turned down. - technical/interpreter.md > Builtins and libraries - the
BuiltinCtxsignature.
M8 - System library namespacing
Status: done.
A hybrid namespace model so domain libraries can ship without
polluting the bare-name pool, plus the first real namespaced
library (os) so the machinery has a non-synthetic exercise.
- Hybrid model. Essential libraries (
io,convert,math,strings, auto-loadedcore) stay flat - their builtins are bare names. Domain libraries register through a new namespaced API (RegisterNamespaced/RegisterNamespacedConst) and are addressed byprefix.name(...)/prefix.NAME. The library’s name doubles as the namespace prefix. - Qualified calls and constants. New AST nodes
QualifiedCallExprandQualifiedConstRefExpr; parsed asIDENT "." IDENT(then(decides). Lookup is keyed by(namespace, name)and gated byuse lib;. use NAME as ALIAS;aliasing. Optionalasclause onuse. Rename-not-addition: afteruse bio as b;onlyb.resolves,bio.foo()errors with a “did you meanb?” hint; the canonical namebiois freed for ordinary identifier use. Matches Python’simport foo as bar. Aliasing is rejected for flat libraries (use math as m;errors as meaningless).- Namespace prefix is a reserved identifier. After bare
use bio;,func bio() {}errors withshadows imported namespace 'bio'. Afteruse bio as b;, onlybis reserved. - No migration. The change is purely additive; all five flat essentials continue to work unchanged.
- Demo library
os(minimal slice). First namespaced library:os.platform() -> string,os.getEnv(name) -> string,os.JENNIFER_LF,os.JENNIFER_OS. Two functions plus two constants - enough to exercise namespaced zero-arg calls, namespaced calls with arguments, namespaced constants, and aliasing end-to-end. Expands in M15.1.
See:
- libraries/os.md - the shipping demo library.
- libraries/index.md - flat vs namespaced policy and the rule for library authors.
- user-guide/imports.md > Namespaced libraries and aliasing -
user-facing reference for
use NAME [as ALIAS];and qualified calls. - user-guide/style-guide.md > Namespaced calls -
spacing convention around
.. - technical/grammar.md - EBNF for
qualifiedCall/qualifiedConstRefand theuse ... as ...shape; AST table entries for the new nodes. - technical/interpreter.md > Namespaced libraries (M8) -
registration API,
nsPrefixes/nsAliasedAwayresolution tables, no-shadowing rule for namespace prefixes.
M9 - Collection operations
Status: done.
Two new namespaced libraries cover the M6-deferred list/map manipulation helpers, a small append sugar shortens the common write pattern, and two follow-on breaking changes tidy up the flat-vs-namespaced split.
listslibrary (use lists;, namespaced).lists.push,lists.pop,lists.first,lists.last,lists.head,lists.tail,lists.reverse,lists.sort,lists.contains,lists.concat,lists.slice. Non-mutating - every function returns a new list.sortaccepts numeric, string, or bool elements (mixed int/float promotes; other mixes error); comparator-based sort is deferred until methods are first-class.mapslibrary (use maps;, namespaced).maps.keys,maps.values,maps.has,maps.delete,maps.merge. Same shape.maps.deleteof a missing key errors (strict at boundaries, matching$m[missing]);maps.mergelayers the second arg over the first.- Sugar:
$xs[] = item;- write-only target meaning “just past the end of the list”. Equivalent to$xs = lists.push($xs, item);. Reads of$xs[]and chained forms ($xs[0][]) are parse errors; non-list targets error at runtime. New AST nodeAppendStmt. - BREAKING:
has()moved fromcoretomapsasmaps.has(m, key). Barehas(...)callers now needuse maps;and the qualified form.haswas the only non-polymorphic name in core;lenstays because it genuinely spans string / list / map. - BREAKING:
stringslibrary moved from flat to namespaced.upper(s)→strings.upper(s),contains(s, sub)→strings.contains(s, sub), etc. across all 15 functions.use strings;itself is unchanged. The M8 library-author rule named exactly these collision-prone verbs (contains,split,replace,join); acting on it now keeps callers off the wrong shape before more libraries arrive. After M9 the remaining flat libraries areio,convert,math, and auto-loadedcore.
See:
- libraries/lists.md / libraries/maps.md - function reference for each new library.
- libraries/strings.md - now namespaced (M9 migration note at top).
- libraries/index.md - updated flat-vs-namespaced catalog and the library-author rule.
- user-guide/imports.md and
user-guide/types-and-values.md > The
$xs[]append sugar - user-facing reference. - technical/grammar.md - EBNF and AST entry
for
AppendStmt.
The next phase splits into four arcs after two architectural
prerequisites: M10 lands the namespace-first library architecture
that the rest of the standard library will be built on; Phase A
(M11-M13) finishes the language so libraries have something to
stand on; M14 closes the lexer-side gap (fmt losing comments
and shebangs) so the first wave of struct-using libraries can
ship with doc-comments intact; Phase B (M15.x) ships the
foundational libraries that every Jennifer program needs,
finishing with M15.8 - the first public release (CI, prebuilt
binaries, .deb / pacman / AUR packaging); Phase C (M16.x) ships
I/O libraries on top of the now-released foundation; Phase D
(M17-M20) ships the higher-level ecosystem (Jennifer-coded
libraries, the module system that unblocks them, crypto, a
server). Everything beyond a 1.0.0 - embedding, WASM, and
specialised domains - lives in the
beyond-1.0.0 idea collection.
The library milestones use sub-numbering (M15.1, M15.2, …) so each library ships and is reviewed independently. This is the first time we use sub-milestones; the practice is justified because each library is small enough to land in a single sitting once the language foundation is in place.
M10 - Namespace-first library architecture
Status: done.
A pre-language-completion API-shape correction: every library is
now namespaced, with bare-name globals reserved as a narrow
core-only exception. Small implementation surface, large API
shape; pre-1.0 is the window for this kind of change.
- BREAKING:
io,math,convertmigrate to namespaced-only.printf(x)→io.printf(x),sqrt(x)→math.sqrt(x), etc. The “io is special, keep it flat” alternative was considered and rejected at kickoff to keep a uniform “every call carries its library name” rule.strings,lists,maps,oswere already namespaced (M9/M8). - BREAKING:
convert’s four conversion callees are renamed toconvert.toInt,convert.toFloat,convert.toString,convert.toBoolso they don’t collide with the type keywords (int,float,string,bool);convert.typeOfkeeps its name. Theto-prefix also reads as English (“convert to int”) at the call site. - BREAKING: file-splice keyword
import→include.include "x.j";is the textual splice; theimportkeyword is reserved for the M17 module system and produces a migration-hint error today. Mixing-mistake diagnostics updated. - BREAKING for embedders: registration API renamed.
Register/RegisterConst→RegisterGlobal/RegisterGlobalConst, making their role explicit (“expose this name globally”). The namespaced API (RegisterNamespaced/RegisterNamespacedConst) keeps its name and is the recommended default. Per-library storage (globalFnsByLib,globalConstsByLib) so two libraries with the same global name can no longer silently overwrite each other at Install time; the resolution map is populated byprocessImportswhen a library activates. mathabsorbs the planned non-crypto random helpers:math.rand(),math.randInt(lo, hi),math.randSeed(n). Three functions don’t justify their own library under the new threshold (next bullet); pseudo-random fitsmath’s pure-numeric charter. The crypto-grade variant still ships in M20.1crypto. The originally planned M14.2randomlibrary is removed.coreis the only library publishing bare-name globals.lenandJENNIFER_VERSIONonly - nocore.len/core.JENNIFER_VERSIONqualified form, because shipping the same name two ways violates stance #1.coreis the auto-loaded escape hatch, and its asymmetric exposure is the whole point.- Three globals-publishing rules in
processImports, all forward-looking (inert today sincecoreis the only globals-publishing library and can’t beused):- Duplicate
useof a globals-publishing library is rejected (library 'X' already in scope); REPL no-ops a repeat. use X as Y;whereXhas globals but no namespaced names is rejected as meaningless.- Two active libraries publishing the same global name are
rejected at the second
use(library "B" collides with already-active library "A" on global "VER"). The pre-M10 flat-only-alias-meaningless check is removed for the general case but kept as rule 2.
- Duplicate
- Library-author guidance updated. The
docs/libraries/index.md“flat vs namespaced” framing is retired; the new policy is “every library is namespaced; onlycoreships globals viaRegisterGlobal.” The “deserves its own library” threshold is raised from M8’s “3+” to “5+ functions or constants”: anything smaller folds into the most-related existing library. The non-crypto random helpers (3 functions) are the first case the new rule caught.
See:
- libraries/io.md, libraries/math.md, libraries/convert.md - migrated library references.
- libraries/index.md - retired flat-vs-namespaced framing; new library-author policy and 5+ threshold.
- user-guide/imports.md -
useandincludekeyword reference; namespaced-call and aliasing rules. - user-guide/types-and-values.md -
convert.toInt/convert.toFloatexample placement in the “explicit conversions” section. - technical/rejected.md - “Methods on structs” (M14.3 trigger; recorded here in M10’s wake because M10’s review touched the same call-shape question) and other related rejected alternatives.
No new language features land here - that’s M11.
Phase A: language completion (M11-M13). These three milestones close the biggest daily-use gaps and add the foundational types every later library needs.
M11 - Control-flow completion
Status: done.
Closes the biggest daily-use gap in the language and rounds out the
printf modifier table at the same time. Five new keywords (break,
continue, repeat, until, exit) and two new printf features.
break;/continue;in every loop kind (while/for/for-each/repeat). Innermost loop only; misuse outside a loop or across a method-call boundary is a positioned runtime error.continuein C-styleforstill runs the step before re-checking the condition (matches C/Go).repeat { } until (cond);post-test loop. New keywordsrepeatanduntil;do { } while ...considered and rejected because the inverted condition is the whole point of switching tountil.exit;/exit EXPR;terminate the whole program (exit code 0 / EXPR-as-int). Distinct fromreturn(method-scoped): skips every caller frame and remaining top-level statement. Implemented as anExitSignalsentinel error the CLI translates into the OS exit status.- Bundled: printf
%s|align=centerrounds out the align set. Rejected on every other typed verb (centred numbers break columnar output). - Bundled: printf
%aaggregate verb for lists and maps (deferred from M7; unblocked by M6 + M9). Modifiers:sep,kv,open,close,depth=N,null=skip. The modifier-list parser was extended with a"..."quoted-value form (%a|sep=", ") so values can contain spaces / reserved characters; standard\n \r \t \\ \"escapes. - Post-dot name relaxation. Reserved words read as identifiers in
the name slot of a qualified call (
strings.repeat,lists.breakif anyone wrote one), preserving thestrings.repeatlibrary function afterrepeatwas reserved as a loop keyword.
See:
- user-guide/control-flow.md -
repeat/until,break/continuescope rules,exitvsreturn. - libraries/io.md -
%amodifier table,%s|align=centerexample, quoted modifier values. - technical/rejected.md -
%a|json=*/%a|xml=*/%a|yaml=*(serialisation modifiers stayed rejected even after%aitself shipped) and thedo { } whileshape for the post-test loop.
M12 - Bytes and bit operators
Status: done.
Adds the buffer-shaped primitive and the bit-twiddling vocabulary the standard library needs for hashing, encoding, crypto, and network code in later milestones.
- New primitive type
bytes- mutable byte sequence; value semantics on assignment / parameter binding; deep-const. Reads yieldintin[0, 255]; writes accept the same range and reject anything else. Append via the existing M9$b[] = byte;sugar.len($b)returns the byte count. - New
convert.bytesFromString(s, codec)andconvert.stringFromBytes(b, codec)- bytes ↔ string codecs. Only"utf-8"today (further codecs ship in M15.7encoding). Invalid UTF-8 input is an error - no silent replacement characters. - Bit operators on
int:& | ^ ~ << >>. Python-style precedence (comparison <|<^<&< shifts <+ -), so$x & 0xff == 0parses as($x & 0xff) == 0.~is bitwise NOT. Shifts are arithmetic; negative count rejected; count >= 64 saturates to 0 / -1.^ships as a primitive operator (CPU primitive with unique algebraic properties - same justification-has against being composable from+and unary-). - Non-decimal integer literals: hex
0xff, octal0o755, binary0b1010_0110._accepted between digits in any base (including decimal1_000_000and float mantissas). Never adjacent to the prefix or another_. Lexer-only change. - Resolves M7-deferred stdin builtins:
io.readBytes(n) -> bytes(exact n; partial at EOF thenio.eof()becomes true) andio.readChars(n) -> string(n runes, UTF-8 decoded). Both compose with M7’sio.eof()unchanged.
See:
- user-guide/types-and-values.md -
bytestype, value semantics, index-write rules. - libraries/convert.md - codec functions, UTF-8 strictness.
- libraries/io.md -
io.readBytes,io.readChars. - user-guide/control-flow.md - bit-operator precedence table.
- user-guide/syntax.md - non-decimal literals + digit separator.
M13 - Structs and catchable errors
Status: done.
The composite-data milestone, batched into two sub-milestones in dependency order: M13.1 ships the struct mechanism, M13.2 ships the error-handling design that uses it. Together they unblock every library that wants composite returns and give the language a recoverable-error story.
M13.1 - Structs / records
Status: done.
def struct Name { field as type, ... };at top level (hoisted before the first statement; duplicate names error inRun, silently redefine in the REPL).- Literals
Name{ field: expr, ... }with every field required;def x as Name;(no init) zero-fills, recursing through nested struct fields. - Field read
$p.field, write$p.field = ...;. Lvalue chains mix[index]and.fieldfreely ($L.from.x = 5;,$bag.items[0] = 99;); index-assign and field-assign share one walker. - Value semantics like lists/maps;
constdeep at any depth. - Strict at boundaries: unknown struct type, missing / unknown field at literal, field-type mismatch on write, field access on a non-struct value are all positioned errors.
See:
- user-guide/types-and-values.md - language angle.
- technical/interpreter.md -
runtime details (
KindStruct, hoisting, unified lvalue walker). - technical/grammar.md -
structDef,structLit,fieldAssign, mixed-tail lvalues. examples/structs.jstandalone;examples/showcase.j=== M13.1 structs ===section.
M13.2 - try / catch / throw
Status: done.
Catchable errors. New keywords try, catch, throw. Depends
on M13.1 because the canonical error value is a struct.
try { body } catch (NAME) { handler }runs the body and, on a catchable error, binds the thrown value to$NAMEin a fresh per-handler scope.throw EXPR;raises any value; convention is the auto-hoistedError{kind, message, file, line, col}struct.- Runtime errors (out-of-bounds, missing key, type mismatch,
etc.) are wrapped into the canonical
Errorshape on entry to the catch (kinddefaults to"runtime"until sites opt in to specific tags); user code catches both kinds uniformly.throw $err;inside a catch re-raises to the enclosingtry. - Not catchable:
exit(program-level escape, propagates throughtry);return/break/continue(control flow, flow throughtryunchanged). - No
finallyand no typed catch in v1. - Internals:
ErrorSignalsentinel parallelsExitSignal;runtimeError.Kindfield threads the symbolic tag; user code may not redefine the auto-hoistedErrorstruct.
See:
- user-guide/control-flow.md - language angle.
- technical/interpreter.md -
runtime details (
ErrorSignal, wrapping, flow passthrough). - technical/grammar.md -
tryStmt,throwStmt. examples/trycatch.jstandalone;examples/showcase.j=== M13.2 try/catch ===section.
M14 - Lexer comment + blank-line preservation
Status: done.
Closes the two M5-deferred items (fmt drops comments, fmt drops blank lines). No language change - the runtime still
never sees comments.
- Lexer emits trivia tokens (
TOKEN_COMMENT_LINE,TOKEN_COMMENT_BLOCK,TOKEN_COMMENT_SHEBANG,TOKEN_BLANK_LINE). Shebang on line 1 col 1 is its own kind; runs of blank lines collapse to one. - Preprocessor and parser strip trivia at entry;
jennifer fmtwalks the raw lexer stream and re-emits trivia via a dedicatedemitTriviapath that doesn’t disturb the surrounding state machine. - Block comments nest via depth counter; unterminated comments
error at the outermost
/*. - Token-level over AST-level: the original spec proposed
AST-attached
LeadingComments/TrailingCommentslots and ajennifer ast --with-commentsflag - dropped in favour of the simpler token-level path. Add them back if a future doc generator needs structured per-statement attachment.
See:
- user-guide/style-guide.md - Comments section (block comments nest; inline-comment spacing exception).
- technical/lexer.md - trivia emission, shebang detection, nesting depth counter.
- technical/cli_fmt.md -
fmt’s trivia re-emission.
Phase B: foundational libraries (M15.x). Small, frequently-used libraries grouped under M15 with sub-numbering. The leading M15.0 slot is the “wrap-up of existing libraries” (extensions to M8 / M9 / M10 libraries that depend on language features added since); later slots ship a new library each. M15.8 closes the phase by making the result installable before Phase C starts adding I/O on top.
M15 - foundational libraries + first public release
Status: done. All nine sub-milestones shipped. Three are language work (M15.2, M15.4), the rest are library / tooling / release work. Two recurring patterns surfaced in the shipped APIs and are worth remembering:
- Codec-table shape (algorithm/format/codec passed as a
string argument). Used by
hash.compute(b, algo),crc.compute(b, algo),encoding.encode(s, codec),encoding.toText(b, format). Originally adopted because Jennifer’s letters-only identifier rule rejects digits in method names (sohash.md5(...)won’t parse), but it also honours stance #1 by collapsing parallel verbs into one. - Integer-handle struct for opaque resources (M15.2’s
namespaced-struct mechanism + a single
id as intfield indexing into a Go-side map). Used byos.Process,hash.Stream,crc.Stream.
M15.0 - existing-library extensions
Done. Two extensions to the M9 lists library that needed
post-M14 language features: lists.shuffle(xs) (Fisher-Yates,
respects math.randSeed) and lists.range(start, end[, step])
(half-open, deliberate single-arg omission per stance #2). See
libraries/lists.md and
technical/design-decisions.md > Half-open ranges
for the half-vs-closed-range rationale.
M15.1 - os + meta (process metadata)
Done. Reshapes the M8-era os surface around one rule:
immutable per-run host facts are uppercase constants
(PLATFORM, ARCH, EOL, DIRSEP, PATHSEP, ARGS),
operations are functions (getEnv, hasFlag, flag). Drops
the JENNIFER_ prefix that only made sense for bare-global use,
and introduces a new meta library for interpreter-self-identity
constants (VERSION, BUILD). CLI forwards trailing args to
os.ARGS (script path at index 0). Breaking renames
(JENNIFER_VERSION -> meta.VERSION, os.platform() ->
os.PLATFORM, os.JENNIFER_OS -> os.PLATFORM,
os.JENNIFER_LF -> os.EOL); old names now produce plain
“undefined” errors with no rename-hint. See
libraries/os.md and
libraries/meta.md.
M15.2 - Language: library-provided namespaced struct types
Done. Language work slotted inside Phase B because the next
wave of libraries (M15.3 os.{Result,Process}, M15.5 time.*,
M15.6 hash.Stream/crc.Stream, future M16.1 fs, M16.2 net)
all need their own struct types and M13.1 only handled bare-IDENT
names. Adds def x as lib.Name; type syntax,
lib.Name{field: ...} literals, and the Go-side
Interpreter.RegisterNamespacedStruct API. Reuses M13.1’s value
semantics, deep-const, and strict-boundary machinery; only the
resolution path differs. User code can’t register structs (Go-side
only); methods on structs and inheritance stay out of scope. See
technical/interpreter.md > Library-provided namespaced structs.
M15.3 - os external-program execution
Done. First library to consume the M15.2
namespaced-struct mechanism. Surface: os.Result {exitCode, stdout, stderr} + os.Process {pid} as the public types;
os.run(argv) -> Result blocking, os.spawn(argv) -> Process
non-blocking, os.wait/poll/kill(p) for handle ops. argv is
always list of string (no shell parsing; explicit
["sh", "-c", $cmd] for that hop). Non-zero exit codes are
values, not errors. TinyGo limitation: TinyGo’s runtime
doesn’t implement os/exec, so the constrained jennifer-tiny
binary returns a friendly “use the default jennifer binary” error
instead of panicking - first place the two-binary story becomes
user-visible. See
libraries/os.md > External programs
and examples/exec.j.
M15.4 - Language: len built-in, core removed
Done. Promoted len(EXPR) from the auto-loaded core library
to a reserved keyword + language primary expression (polymorphic
over string / list / map / bytes). Deleted internal/lib/core/
entirely; use core; now returns a friendly migration error
pointing at the built-in and at meta.VERSION / meta.BUILD.
Stance #2 (“explicit over implicit”) now applies uniformly: every
library name lives behind a use NAME; prefix, no exceptions. See
technical/design-decisions.md > len is a language built-in.
M15.5 - time
Done. One opt-in library spanning instants, durations,
fixed-offset zones, strftime format/parse, and ISO 8601
round-trip. Three namespaced structs: time.Time {nanos, offset}
(fields private API), time.Duration {nanos}, and
time.Zone {offset, name} (fields public, so an IANA / DST
companion can build them). Granularity (date-only
vs time-of-day-only) is a property of formatting, not the value
type. Unix timestamps are constructor / accessor pairs, not a
separate type. IANA names and DST are out of the fixed-offset
core - a Go-backed time-library extension, not a .j data map
(see the Long-horizon “time: IANA / DST zones” entry). Three
sub-milestones: M15.5.1 core type + Unix + calendar + 1-based
ISO weekday + arithmetic / comparison; M15.5.2 strftime
format/parse (chosen over Go’s reference-time style for
familiarity; v1 verbs %Y %m %d %H %M %S %z %a %A %b %B %j %u %%) + time.zone(offset, name) + time.inZone + the time.UTC
constant coexisting with the time.utc() function via
case-sensitive lookup + time.iso / time.fromIso RFC 3339
round-trip; M15.5.3 the examples/benchmark.j side-by-side
TinyGo-vs-Go workload suite (eight workloads; the original spec’s
“Sieve of Eratosthenes” became trial-division because Jennifer’s
value-semantic list mutation turns the sieve into O(N^2)). See
libraries/time.md,
examples/time{,-format,benchmark}.j.
M15.6 - hash and crc
Done. Two opt-in libraries with parallel surfaces: hash
for cryptographic-style digests ("md5", "sha1", "sha256"),
crc for non-cryptographic checksums ("crc32" IEEE, "crc64"
with Go’s crc64.ECMA polynomial). Output is raw bytes
(big-endian 4 / 8 bytes for CRC, natural width for hash). The
split keeps “transport integrity” vs “content addressing”
visible at the import line and matches Go’s crypto/* vs
hash/crc* arrangement. Both libraries ship the codec-table
shape: compute(b, algo) one-shot, stream(algo) +
update($s, $b) + finalize($s) -> bytes for chunked input.
Streaming reuses the
integer-handle pattern from os.Process.
No convenience wrappers like hash.md5String or hash.computeHex
(stance #1; users compose convert.bytesFromString and
encoding.toText). Struct hashing deferred (needs stable byte
serialization, its own design problem). See
libraries/hash.md,
libraries/crc.md, examples/hash.j.
M15.7 - encoding
Done. Three-part surface: introspection
(isAscii/lenBytes/lenRunes), binary-to-text
(toText/fromText for "hex"/"base64"/"base64-url"), and
character codecs (encode/decode/codecs). The cross-kind
UTF-8 pair stays in convert (M12); encoding owns the
table-based codec proliferation. Four codecs ship: "ascii",
"iso-8859-1", "windows-1252", "ebcdic". The spec’s per-format
verbs (encoding.hex, encoding.base64, …) consolidated into
the codec-table pair to dodge the same digit-in-identifier rule
M15.6 hit. Codec names and format strings are exact-match (the one
canonical spelling only; the original alias / case normalisation was
later dropped as a strictness lift, stance #2).
Windows-1252’s five canonically-undefined positions (0x81, 0x8D,
0x8F, 0x90, 0x9D) reject symmetrically on encode and decode.
Long-tail single-byte codecs (ISO-8859-{2..16},
Windows-{1250,1251,1253..1258}) later shipped in
M16.15, generated from the Unicode
mapping files. See
libraries/encoding.md,
examples/encoding.j.
M15.8 - distribution + first public release
Done. Packaging / CI / release-engineering only; no
.j-source language change. Four sub-phases:
- CI (
.github/workflows/test.yml,release.yml). PR gate runsgo vet+gofmt+go test ./...+make build+ per-binary smoke run + repo-wide em-dash scan. Release triggers on bare-semver tags ([0-9]*.[0-9]*.[0-9]*, novprefix per project convention), cross-compiles both binaries forlinux/{amd64,arm64}from one runner, QEMU-smoke-tests the non-native arch, runs the benchmark on amd64 so release notes carry fresh numbers, publishes a draft Release. - Packaging under
packaging/{debian,arch,mime,man}/.scripts/build-deb.shproduces the.deb(binaries + gzipped man pages + sharedtext/x-jenniferMIME definitionupdate-mime-databasehooks). AUR shipsPKGBUILD-bin(release tarball) andPKGBUILD-git(source-tracking) with a shared.installhook. Release pipeline auto-fillsPKGBUILD-bin(realpkgver+ realsha256sums_*) as a release asset so the AUR push is a one-stepcurl. The.pacmanstandalone artefact from the original spec was dropped -PKGBUILD-bin+makepkgcovers the same need.
- Docs site via mdBook 0.5.3 (pinned, fetched via direct
curl from
rust-lang/mdBookreleases - no third-party action).book.tomlat repo root,src = "docs",docs/SUMMARY.mdmaps the existing tree into five parts,docs/introduction.mdis the docs-site landing (README stays GitHub-repo-focused)..github/workflows/docs.ymlpublishes to GitHub Pages on every push tomain. - User-facing install docs. README gained “Which binary?” +
“Install” sections with one command per path.
installing.mdrestructured to put package paths first; build-from-source positioned as the developer path.RELEASE.mdat the repo root documents the steps CI can’t do (AUR SSH push, draft-publish review, pre-tag readiness checks).
Conventions established (worth keeping):
- Bare semver tags (
0.14.1, novprefix); all pipelines pass the tag straight through. - No top-level
LICENSEfile in v1 - the LGPL-3.0 text ships inline inpackaging/debian/copyright(the form distros actually consume) + README links to gnu.org’s canonical URL.
One-time manual setup before the first push to main: in
GitHub repo Settings -> Pages, set “Source” to “GitHub Actions”
so docs.yml can deploy.
Deferred out of this milestone (not gated on it): the cross-build for macOS / Windows and a real apt repository stay in Requirements for 1.0.0 stable; the extra distribution formats (Homebrew, Snap, Nix, Flatpak, AppImage) moved to the horizon idea collection.
M16 - I/O libraries and developer tooling
Status: done. Phase C: system libraries that touch the OS or do significant compute, opened by a concurrency primitive (M16.0) that the I/O libraries build on, then a developer-tooling trio (lint / profile / test) and a run of self-contained data libraries. All sub-milestones shipped; git history holds the full per-milestone specs.
M16.0 - Lightweight concurrency
Done. spawn { ... } (block primary expression), task of T (new
compound kind), and the task library (wait / poll / discard /
waitAll / waitAny). Goroutine-backed but data-race-free by
construction: snapshotForSpawn deep-copies a two-frame globals+locals
snapshot at launch (tasks are the one carve-out from value semantics -
copies share the TaskState pointer). A per-run registry loud-fails
unobserved task errors at exit and bumps the exit code (a non-terminating
undiscarded spawn hangs at exit - the documented trade-off). task.wait
re-raises a body error at the wait site for try/catch; waitAny is
the runtime’s only reflect.Select. The Makefile passes
-stack-size=2mb -scheduler=tasks to TinyGo. See
concurrency.md, task.md.
M16.1 - fs
Done. Blocking filesystem I/O composed with spawn (no *Async
variants): whole-file read/write/append (String/Bytes), metadata
(exists/isFile/isDir/stat -> fs.Stat), directory ops with the
two-verb recursion split (mkdir/mkdirAll, remove/removeAll, plus
rename/list/walk), and buffered fs.File handles
(open/readLine/…/close; eof peeks one byte). Path- vs
handle-form verbs dispatch on first-arg kind; fs.File shares state
across copies (the handle carve-out from value semantics). See
fs.md.
M16.2 - net
Done. TCP (connect/listen/accept/readBytes/writeBytes),
UDP (listenUDP/sendTo/recvFrom), and DNS
(lookup/reverseLookup); polymorphic close/address over three
handle registries. Blocking calls compose with spawn
(accept-loop-per-connection). Build-tag split: jennifer-tiny returns
friendly-error stubs (no netdev in TinyGo). See net.md.
M16.3 - regex
Done. RE2 (Go regexp, linear-time) over strings:
matches/find/findAll/replace/split/escape;
regex.Match{text,start,end,groups,groupsNamed} with rune indices and a
start=-1 no-match sentinel; an implicit 128-entry LRU pattern cache.
Full surface in both binaries. See regex.md.
M16.4 - testing (system-library primitives)
Done. The irreducible system-side surface a .j test framework needs:
testing.run(name) invokes a zero-arg user method via the new
Interpreter.CallByName, times it, and classifies every failure mode
into a testing.Result; results/reset manage a mutex-guarded
accumulator; report renders text / TAP / JUnit. The one place exit is
caught (at the Go level, so language try/catch still cannot). See
testing.md.
M16.5 - Interpreter performance pass
Done. Five sub-milestones, user-visible behaviour unchanged: .1
shared-marker copy-on-write on compound Values (append-in-a-loop O(N^2)
-> amortised O(N)); .2 parse-time lexical slot resolution
((Depth,Slot) coordinates + a slots slice, undefined/shadowing promoted
to parse-time errors); .3 pooled + pre-resolved + slot-bound
method-call frames; .4 namespaced-call / comparison / arg-bind /
root-cache fast paths; .5 compile-time constant folding plus a
Share() scalar fast path. Numbers in tinygo.md.
M16.6 - Developer tooling: linting
Done. jennifer lint flags compile-legal-but-suspect patterns:
grouped IDs (L0nn source errors, always on / L1nn correctness /
L2nn style / L3nn lifecycle), # lint-disable[-file]: IDS
suppression, --checks / .jennifer-lint config, and human / JSON /
GitHub output (source errors render in the chosen format, so a JSON
pipeline stays valid); exit 0/1/2. !tinygo. See
cli_lint.md.
M16.7 - Developer tooling: profiling
Done. jennifer profile attributes work to .j source positions
(what go tool pprof cannot): a statement profile (hit count +
self/cumulative wall-clock) and an --allocs value-copy profile; table /
pprof (hand-encoded gzipped protobuf) / Chrome-trace output; program
output goes to stderr so the profile owns stdout. !tinygo. See
cli_profile.md.
M16.8 - Testing framework consolidation
Done. An assertion vocabulary on M16.4’s primitives (assertEqual
… assertThrows, throwing Error{kind:"assertion"} at the call site),
CallByNameWith/runWith argument dispatch, and the jennifer test
subcommand (test* discovery or --filter, setUp/tearDown,
--isolated per-test subprocess, text/TAP/JUnit, exit 0/1/2). Builtins
can now raise a catchable Jennifer error via interpreter.RaiseError. See
cli_test.md.
M16.9 - json
Done. Hand-rolled RFC 8259 encode/decode onto the tagged-union Value
(no encoding/json, no reflect). encode/encodePretty/decode;
structs and map of string to V map to objects, bytes to base64,
numbers to int when integral else float. Also closed a type hole: a
generic collection (a fresh literal or decode result) is validated entry
by entry against the declared element type at every binding boundary.
Decode’s return shape was later superseded by
M16.16. See json.md.
M16.10 - uuid
Done. RFC 9562: uuid.generate("v4") (random) / generate("v7")
(time-ordered), the version tag a string argument (identifiers are
letters-only), plus parse/isValid/version and constant NIL.
Randomness draws from math’s shared seedable RNG (documented
non-crypto; swaps when M20.1 crypto lands). See uuid.md.
M16.11 - compress
Done. Byte-stream size reduction (distinct from encoding’s
representation codecs): pack/unpack for "gzip"/"zlib"/"deflate"
with an optional "fast"/"default"/"best" level, plus a streaming
compress.Stream handle. Go compress/*, TinyGo-clean. See
compress.md.
M16.12 - archive
Done. tar / zip containers over bytes (no fs, value-semantic):
pack/unpack (verbs shared with compress) for
"tar"/"zip"/"tar.gz"; a bundle is a
list of archive.Entry{name,data,mode,mtime}. Go
archive/tar+archive/zip. See archive.md.
M16.13 - os.isTerminal
Done. os.isTerminal(stream) ("stdout"/"stderr"/"stdin") ->
bool, the gate for ANSI colour, via the character-device mode bit
(os.ModeCharDevice) - pure stdlib (keeps x/term CLI-scoped),
TinyGo-clean; an unstattable stream reports false. See
os.md.
M16.14 - net TLS
Done. net.connectTLS(address) (implicit TLS) and
net.startTLS(conn) (in-place STARTTLS upgrade), both yielding the
transport-agnostic net.Conn. Certificate verification is on by default,
with a net.TLSOptions{skipVerify as bool, caCert as bytes} opt-out. Go
crypto/tls on the !tinygo build (stubbed on jennifer-tiny). See
net.md.
M16.15 - encoding completion
Done. Filled out encoding: toText/fromText gained
quoted-printable, base32/base32-hex, ascii85, and z85; the full
ISO-8859-{1..16} / Windows-{1250..1258} single-byte codecs, generated
from the Unicode mapping files (gen_codecs.go -> codecs_gen.go) so
only ascii/ebcdic stay hand-written. Format and codec names are
exact-match (the normalisation layer was dropped as a strictness lift,
stance #2). See encoding.md.
M16.16 - json.Value
Done. The strict home for heterogeneous JSON without a language top
type: json.decode returns an opaque json.Value - the first
KindObject (the opaque sibling of KindStruct: discriminated by
(namespace, name), minted only by a library, rejecting operators /
[i] / .field). convert.typeOf reports "object",
convert.objectType the specific "json.Value". Reads and non-mutating
writes share JSON Pointer (RFC 6901) addressing -
typeOf/get/has/keys/length/as*/isNull and
map/list/set/insert/append/remove/move (strict / no-vivify,
- end-marker), with node types in list/map vocabulary - and a
displayer hook renders a handle as its JSON. json.decode’s return type
changed (a pre-1.0 break) and the decoder’s number grammar was tightened
to json.org. No any keyword (rationale in
rejected.md). See json.md.
Phase D: higher-level and Jennifer-coded libraries (M17-M20).
M17 - Module system for Jennifer-coded libraries
Status: done. All six sub-milestones shipped. Jennifer-coded libraries
now get their own namespace, scope, and explicit exports via a real
module boundary (import "x.j" as x;, parser + interpreter) that lives
beside the textual include "x.j"; splice (preprocessor, for composing one
module out of several files). The settled, cross-cutting decisions
(turned-down alternatives in rejected.md): import
is a parser + interpreter feature, not a preprocessor splice; each module is
its own resolution context (own use set, namespace + export tables); the
module top level is declarations-only - no mutable module state, so
spawn capture is unaffected; the one global Error stands (modules add
distinctly-named error structs, never redefine it); private by default,
a leading export publishes a name (no public / private keyword);
multi-file modules assemble via include behind one entry file (no
directory-as-module, no cross-file re-export); modules need a filesystem
(an FS-less jennifer-tiny host fails with the ordinary search-path error).
User-facing reference: imports.md +
modules/index.md. M18.x builds on top.
M17.1 - Source tree and resolution
Done. Path resolution in internal/module: Classify + Resolve map
an import path (local ./ / ../, absolute /, or a bare name walked on
the search path) to a canonical absolute path, rejecting a name found in
two search dirs and a not-found. The system module directory resolves
--sysmoddir > JENNIFER_SYSMODDIR > compile-time default (surfaced as
meta.SYSMODDIR; a named CLI / env dir that is missing or not a directory
refuses to start, the compile default is best-effort), and -I DIR
(repeatable) appends to the search path after it. jennifer version -v
reports the resolved layers. See cli.md.
M17.2 - Import statement and loader
Done. import "path.j" [as NAME]; is a real statement
(ModuleImportStmt) - the preprocessor passes it through, the parser
builds the node. The loader (internal/interpreter/module.go) runs each
module in a fresh sub-interpreter sharing one moduleReg (run-once cache
by canonical path, in-progress load stack, search path), so run-once,
depth-first post-order init, and cycle detection (erroring with
every edge named) all fall out of the recursion. Load-time errors (a parse
error or a throwing def const initializer) fail the program and are not
catchable - an import is a declaration, not an expression, so it cannot
sit in a try / block. jennifer fmt / ast / tokens round-trip an
import line. See imports.md.
M17.3 - Module scope and namespacing
Done. A module top level is declarations-only
(checkModuleDeclarationsOnly: only def const / def struct / func /
use / import; a mutable def or free-standing statement is a
positioned load-time error - scripts keep both). loadModuleImports binds
each alias (the as NAME, or the file stem) into moduleAliases,
collision-checked against library prefixes. Consumer resolution rides the
qualified-reference eval layer: evalQualifiedCall / evalQualifiedConst
dispatch alias.fn(args) into the module’s own interpreter via
CallByNameWith (arguments evaluated in the consumer, body run against the
module’s globals + methods) and read alias.CONST from its scope. use
non-transitivity, run-once sharing, and -race safety all follow from the
fresh-sub-interpreter-per-module model - a module holds only immutable
constants and read-only methods. See interpreter.md.
M17.4 - Exports and visibility
Done. export (a keyword) publishes a top-level func / def struct /
def const; unmarked names stay private (reaching mod.helper from
outside is a positioned “not exported from module” error), and export in
a jennifer run script is rejected (module vs script by entry, via the
isModule flag). checkReferentialClosure rejects an exported struct
field or exported function parameter typed as a private module struct;
library / namespaced types cross freely. Cross-module struct identity
is boundary translation (retagStructs): a module’s structs are bare
inside it and re-tagged to (module-stem, name) as a value crosses out to
an importer and back on the way in, so def p as mod.Point,
mod.Point{...}, field reads, and pass-back all type-check while a.Point
and b.Point stay distinct. The retag walks values and the collection
type tags a list / map carries (retagType), so a
list of mod.Point handed back into a module reads as its bare
list of Point parameter. A co-located MODULE_test.j overlay (a token
splice in jennifer test) runs white-box tests against the module’s
private names.
M17.5 - ansi module
Done. modules/ansi.j - terminal styling as explicit string wrappers,
the first module built on the system (pure Jennifer, one use os;
dependency across the boundary; a real end-to-end dogfood of import /
export / resolution). Exports color / bgColor / style (bold / dim /
italic / underline / reverse) / rgb truecolor / strip, plus per-colour
and per-style shortcuts (ansi.red(s), ansi.bold(s), …). The ESC byte
is built from a one-byte bytes (no string-literal escape for it); strip
uses regex; unknown colour / style names throw. Stateless and
TTY-aware: enabled() re-reads NO_COLOR / FORCE_COLOR /
os.isTerminal("stdout") per call, so redirected output stays clean and no
toggle state is stored (honours the no-mutable-state rule); degrades to
always-on when os.isTerminal is absent. Colour is a string wrapper, so a
%s|color= printf modifier is rejected in its favour
(rejected.md). Demo
examples/modules/ansi_demo.j; coverage
internal/interpreter/module_ansi_test.go + a white-box
modules/ansi_test.j overlay reading ansi’s private tables; reference doc
modules/ansi.md.
M17.6 - semver module
Done. modules/semver.j - strict SemVer 2.0.0 as
a second pure-Jennifer reference module (no system dependency beyond
use strings; use convert; use regex;, so it runs on both binaries), and
the foundation the future jvc package manager
(Long horizon) needs since
install gotify>=1.0.0 is semver comparison at its core. Exports
Version { major, minor, patch, prerelease, build } plus parse (throws
on invalid) / isValid / toString (round-trips parse); compare /
lt / eq / gt (SemVer precedence: numeric core, a prerelease ranks
below its release, prerelease fields compared numeric-below-alphanumeric,
build metadata ignored); isStable (no prerelease and major >= 1;
0.y.z is unstable by convention) / isPrerelease; incMajor / incMinor
/ incPatch (reset the lower parts, clear pre + build); and sort (own
pass over compare, since lists.sort is scalar-only). Strict, not a loose
parser: a looser N-segment form (1.2.3.4) has no defined ordering and is
rejected. parse matches the canonical anchored SemVer RE2 pattern with
named groups (regex.find + groupsNamed); the precedence comparison and
sort are hand-written Jennifer - the algorithmic dogfood. meta.VERSION is
valid strict semver, so the module parses Jennifer’s own version. Range /
constraint matching (^1.2.0, >=1.0.0, ~1.2.3) is the harder parser and
is deferred to (or just before) jvc. Building the demo surfaced and fixed
a latent M17.4 boundary gap: passing a consumer-typed list of semver.Version
back into a module list of Version parameter (semver.sort) failed because
the retag re-tagged the struct element values but not the list’s
element-type metadata - now covered by retagType (regression
TestModuleListOfStructCrossesBoundary). Demo
examples/modules/semver_demo.j; white-box modules/semver_test.j overlay
(12 tests); reference doc modules/semver.md.
M18.x - Jennifer-coded modules
Built atop the existing system libraries. Each one ships as a Jennifer
module under modules/ (the directory introduced in M17); none of
them are compiled into the interpreter binary. Sub-milestones in priority
order.
M18.1-M18.40 - shipped modules (compacted)
All done. Forty sub-milestones (with their nested parts) shipped as
pure-Jennifer modules/ (except where noted as a Go system library), each
with the standard discipline: a 100%-passing *_test.j overlay, a
cmd/jennifer/*_test.go integration test, a docs/modules/*.md reference, an
examples/modules/*_demo.j, and catalog / README / JENNIFER.md entries.
Per-function detail lives in docs/modules/; this table is
the milestone-number index (numbers were assigned in rough priority order).
| M# | Module(s) | Surface |
|---|---|---|
| M18.1 | csv | RFC 4180 parse / format (+ *With for any delimiter), header-keyed toRecords / fromRecords. |
| M18.2 | htmlwriter | build an HTML element tree and render escaped HTML5 (element / text / raw / render). |
| M18.3 | markdown | Markdown -> HTML. |
| M18.4.1/.7 | mime | RFC 5322 / 2045 message build + parse, incl. RFC 2047 encoded-words. |
| M18.4.2/.4 | smtp / pop / imap | mail send + POP3 / IMAP receive over net (plaintext / STARTTLS / implicit TLS). |
| M18.4.5/.6 | sasl / idna | SASL auth encoders (incl. XOAUTH2); Punycode / IDNA domains. |
| M18.5 | redis | RESP2 client over net. |
| M18.5.1 | resque | Resque-wire-compatible background jobs on redis. |
| M18.6 | memcache | memcached text-protocol client over net. |
| M18.6.1/.2 | session / ratelimit | server-side sessions + fixed-window rate limiting on memcache. |
| M18.7 | http | HTTP/1.1 client over net (https:// via TLS). |
| M18.7.1/.3 | gotify / rest / oauth | push notifications; ergonomic REST layer; OAuth2 get-a-token - all on http. |
| M18.8 | toml (library) | RFC TOML 1.0 encode / decode; opaque toml.Value, JSON-Pointer walk. TinyGo-clean. |
| M18.9.1 | httpd (library) | HTTP/1.1 server engine over net/http; pull-loop accept / respond. |
| M18.9.2 | web + jennifer serve | .j routing framework over httpd (routes by handler name, :param, middleware, web.Context), dispatched by meta.callMain; serve runs / --watch-reloads a program. |
| M18.10 | flatdb | file-backed JSON document store over json + fs; JSON-Pointer query / edit; crash-atomic save. |
| M18.11 | gpio | Linux GPIO over the sysfs / character-device interface. |
| M18.12 | docblock | the Jennifer doc-comment format + parser (FileDoc tree, drift diagnostics). |
| M18.13 | mqtt | MQTT 3.1.1 pub / sub client over net. |
| M18.14 | prometheus | metrics exposition (produce) + retrieval (query the HTTP API). |
| M18.15 | label | industrial label printing: build / render (ZPL + cab JScript) / emit pipeline. |
| M18.16 | web cookies + sessions | cookie helpers + cookie-keyed sessions on the web framework. |
| M18.17 | totp | RFC 6238 TOTP: generate / verify / uri. Over hash.hmac + encoding + time. |
| M18.18 | webhook | GitHub X-Hub-Signature-256 HMAC sign / verify (pure) + send (over http). |
| M18.19 | bucket | S3-compatible object storage over http (AWS SigV4): connect / get / put / delete / listObjects. One module for AWS S3 + MinIO / R2 / B2. |
| M18.20 | dotenv | .env config: parse / read / load (into env via os.setEnv). Over fs + strings + os. |
| M18.21 | cron | parse cron expressions; next(schedule, after) / matches. A calculator over time. |
| M18.22 | log | leveled structured logging (debug..error; text / logfmt / json) to stdout / stderr / file / RFC 5424 syslog. |
| M18.23 | ical | iCalendar (RFC 5545) build + parse: a Calendar of VEVENTs, escaped + line-folded, dates through time. |
| M18.24 | vcard | vCard (RFC 6350) contacts build + parse; shares the content-line codec (ical_vcard_shared.j) with ical. |
| M18.25 | jsonl | JSON Lines (NDJSON): encode / decode + whole-file + streaming Reader, over json + fs. |
| M18.26 | ipnet | IPv4 / IPv6 addresses + CIDR math: parseAddress / toString (RFC 5952) / parse / contains / netmask / broadcast. |
| M18.27 | ntp | SNTP network-time client over UDP: query / queryWith -> Result (server time + clock offset + round-trip delay). |
| M18.28 | statsd | fire-and-forget StatsD metrics over UDP (count / gauge / timing / set); the push counterpart to prometheus. |
| M18.29 | influxdb | InfluxDB 1.x client on http: line-protocol Point builders + write; query -> parsed Series. |
| M18.30 | slack / discord | incoming-webhook chat notifiers on http: plain send + Block Kit / embed builders (sendMessage). |
| M18.31 | telegram | Telegram Bot API on http: sendMessage / sendPhoto / getMe, getUpdates long-poll (stateful receive loop). |
| M18.32 | websocket | RFC 6455 client over net (ws:// / wss://): handshake + masked send / receive (auto-pong, fragment reassembly). |
| M18.33 | amqp | AMQP 0-9-1 client for RabbitMQ over net: handshake, declareQueue, publish, get (Basic.Get pull), ack. |
| M18.34 | multipart | multipart/form-data (RFC 7578) build + parse (binary-safe); web.multipartForm pairs it with web. |
| M18.35 | pdfwriter | generate PDF documents (text / lines / rects, Standard-14 fonts, FlateDecode via compress); byte-identical output. |
| M18.36 | bloom / ringbuffer | data structures: a Bloom filter (probabilistic set) + a fixed-capacity ring buffer (bounded FIFO). |
| M18.37 | tengine | a text/template-subset engine over a json.Value tree (if / range / with / pipes / layout inheritance). |
| M18.38 | barcode | QR (Reed-Solomon over GF(256), masking, versions 1-10) + 1D (code128 / code39 / ean13 / ean8 / itf); SVG / PNG / terminal. |
| M18.39 | mikrotik | MikroTik RouterOS API client over net: sentence-based binary framing, talk / print / run, plaintext + MD5 login. |
| M18.40 | password | password generate / validate / score against a policy Schema; entropy-based complexity (non-crypto RNG). |
Enabling changes these modules pulled into the system side (each documented under its library):
net.setDeadline- a read/write deadline for socket timeouts (M18.13; later extended to UDP sockets forntp, M18.27).io.eprintf- the stdout-printftwin that writes to stderr (a newInterpreter.Err/BuiltinCtx.Errwriter), the stderr sinklogbuilds on (M18.22).tomlandhttpd- two new Go system libraries (a char-by-char TOML parser and anet/httpserver engine both belong in Go, not a.jmodule); M18.8 / M18.9.1.meta.callMain/meta.definedMain- resolve a method against the entry program (retagging module-own struct args across the boundary), the capability thewebframework dispatches handlers through (M18.9.2).hash.hmac(RFC 2104) and thesha512digest - the HMAC primitivetotp/webhookbuild on (and thatjwt/ SigV4 will reuse).
M19 - cross-cutting tooling
The catch-all bucket for milestones that improve the interpreter or its
tooling but belong to neither the Jennifer-coded modules of M18 nor the Go
system libraries of M20. Numbered sub-entries land here as needs arise.
M19.1-M19.5 are a correctness / performance hardening pass over the
interpreter core and libraries (races, dead code, algorithmic complexity,
resource bounds, module identity); M19.6 is the coverage tool; M19.7 lands the
@scope/package vendored-module resolver; M19.8 is the one-time relocation to
the jennifer-language org and a host-independent vanity module path. None of this
work needs reflect or breaks TinyGo-cleanliness. The smallest,
localized crash / correctness fixes (out-of-range httpd.respond status,
truncated-toml-date-time panic, io.sprintf("%d", MinInt64), math.randInt
span overflow, floorDiv large-quotient garbage, the constant-folder’s
above-2^53 comparison divergence, the missing stream-registry mutexes, the
TaskState.Observed atomic, the numeric-conversion and read-length caps, the
cross-module struct-definition lookup so def x as alias.Struct; zero-value
construction and $x.field = ... writes resolve an imported module struct)
land as they surface rather than waiting on a milestone; each ships with a
regression test.
M19.1 - Interpreter concurrency-safety
Done. Both interpreter data races fixed, each pinned by a -race
stress test (nested-spawn global mutation; eight spawn workers each declaring an
aliased module/library struct in a loop). snapshotForSpawn now snapshots the
launching goroutine’s own root frame via effectiveGlobal(env) instead of the
live i.global, so a nested spawn no longer races the main goroutine’s global
writes. Declared struct types are stamped once, single-threaded, before any
statement runs (resolveDeclaredTypesOnce, after loadModuleImports) and carry
a parser.Type.Resolved marker, so the per-execution re-resolve in execDefine
is a read-only no-op: a def x as alias.Struct reached from concurrent
goroutines never re-stamps the shared AST node. Error timing is unchanged (the
stamp pass is best-effort; an unresolved type still errors at execution at its
original position), and the marker also fixes a latent bug where an aliased
library struct re-resolved in a loop hit the “canonical is aliased” rejection.
Two data races in the interpreter core that the race detector catches and that
can crash a program using nested spawn. Both are small, localized fixes plus
a go test -race nested-spawn stress test.
- Nested-spawn global snapshot race.
snapshotForSpawn(internal/interpreter/interpreter.go) always iteratesi.global.vars- the live top-level frame - regardless of which goroutine launches thespawn. Aspawnnested inside aspawnbody snapshots on a background goroutine while the main goroutine keeps defining / assigning globals: Go’s fatal “concurrent map iteration and map write” (uncatchable, takes the interpreter down). Snapshot fromeffectiveGlobal(env)(the launching goroutine’s own root frame - inside a task, the outer spawn’s snapshot) instead of the livei.global. This is also more correct: a nested spawn captures its enclosing scope, not the main goroutine’s live globals. - Runtime AST mutation.
resolveDeclaredStructNS(internal/interpreter/module.go) writest.StructNS = <canonical>into the shared AST node every time adef x as alias.Struct;statement executes (once per loop iteration, say). The samedefrun concurrently from a spawn body and the main goroutine is a write-write race on the AST node. Resolve declared module-struct types once, at load / resolve time, not per execution.
Acceptance. A nested-spawn stress program that mutates globals on the main
goroutine while inner spawns launch runs clean under go test -race; a
def alias.Struct in a loop body inside a spawn is race-free.
M19.2 - Value representation cleanup
Done. The inert copy-on-write machinery is gone: Value.shared, Share(),
Ensure(), ensureCOW, and the per-VarExpr-read Share() call are removed;
the four mutation sites now grow the binding’s own backing in place, and reads
return the binding value directly. Value semantics rest (as they already did)
on eager deep copies at every store site, documented on the Value type and in
docs/technical/interpreter.md; the write-through alternative is recorded in
docs/technical/rejected.md. The now-dead COW-detachment reporting was stripped
from the --allocs profiler (interface, collector, table, pprof) so it no
longer advertises a section that can never fire. A fresh list / map / struct
literal RHS is already private, so execDefine / execAssign skip the
redundant whole-value copy (rhsFreshLiteral) - proven by a profiler-backed
test (literal binding records zero eager copies; an aliasing def b init $a
records one) alongside a value-independence test; value_alias_test.go and the
full suite (incl. -race) stay green. Value shrinking stays deferred.
The copy-on-write marker protocol added for the append-in-a-loop optimization
is inert: Value.Share() has a value receiver, and Environment.Get /
GetAt return the binding’s Value by value, so the shared flag is set on a
throwaway copy and never reaches the stored binding. Ensure() / ensureCOW
therefore never detach (the whole Share / Ensure machinery is dead code), and
every read of a compound variable heap-allocates a fresh *bool that goes
nowhere - pure overhead in hot loops. Correctness never depended on the
protocol: it rests entirely on the eager deep copies at every binding site
(def / assignment / parameter bind) and on builtins copying before they
mutate.
- Delete the dead protocol. Remove
Share()/Ensure()/ensureCOW, theValue.sharedfield, and the per-readShare()call, and document the eager-copy invariant that actually provides value semantics. The write-through alternative (store*Bindingso the marker propagates and mutation sites genuinely detach) was considered and rejected - aliasing-heavy code is rare precisely because we eager-copy, so the complexity buys little; record it indocs/technical/rejected.md. - Stop double-copying literals.
evalListLit/evalMapLitCopy()every element of a freshly-built literal, thenexecDefine/execAssigneager-copy the whole result again. A fresh literal (or a call result) cannot alias a binding - only Var / Index / Field reads can - so the binding site can skip the copy for non-aliasing RHS shapes, removing two full deep copies per literal binding.
Shrinking Value itself (moving the compound payload behind one pointer so the
scalar case stays small) is deferred: a large cross-cutting churn, and the
map hash index (M19.3) is the bigger algorithmic win. Revisit only if benchmarks
still show Value copying dominating after M19.3.
Acceptance. The Share / Ensure machinery and the shared field are gone,
the alias-stress tests (value_alias_test.go) still pass (value semantics
intact), and a compound-var read in a hot loop no longer allocates; a literal
def binding does one deep copy, not three.
M19.3 - Runtime performance: maps and the call / loop hot path
Done. Maps gained an advisory hash index (Value.mapIdx, encoded scalar key
-> position) guarded by a len(mapIdx) == len(Map) stamp, so $m[$k] = $v over
N keys is O(N) not O(N^2) while insertion order and value semantics are
untouched - any stale / duplicate-key / non-hashable-key case fails the stamp
and falls back to the (correct) linear scan. A 100k-key build plus a 100k
for-each of indexed reads runs in under a second where the quadratic path took
minutes; pinned by map_index_test.go (order, updates, misses, duplicate and
non-hashable keys, value-semantics independence, 5k-key consistency). The
call/loop batch landed too: execForEach and execFor borrow their frames from
envPool instead of allocating per iteration / per loop; DefineAt skips the
enclosing-scope shadow walk on the resolver-verified slot path; Run pre-sizes
i.global’s slots from prog.NumGlobals (no one-at-a-time O(n^2) growth); the
three mutation sites (execIndexAssign / execAppend / execFieldAssign) fetch
and write the root binding through (Depth, Slot) (getBindingRoot /
assignRoot, guarded to keep the REPL name path chain-walking); and
lists.reverse / head / tail / slice / concat take a shallow struct copy
instead of deep-copying the whole argument they immediately overwrite. Full suite
(incl. value_alias_test.go, the map ordering tests, and -race) stays green on
both toolchains.
The biggest algorithmic issue in the runtime plus the call- and loop-overhead
batch. None need reflect or break TinyGo-cleanliness.
- Maps as association lists (biggest win).
Value.Mapis a[]MapEntry; index reads (indexInto) and writes (writeIndexedSlot) linear-scan with a recursiveValue.Equalper entry, and map equality is O(n*m) - building a map with$m[$k] = $vin a loop is quadratic. Maintain a side hash index (map[string]intover a canonical scalar-key encoding) alongside the ordered slice (insertion order stays a language guarantee), falling back to the linear scan for the rare non-hashable key. - Call / loop hot path.
execForEachallocates a freshNewEnvironmentSized(new name map) per iteration andexecForan unpooled header env - both should borrow fromenvPoollikeexecBlock.DefineAtre-runsexistsInChainand mirrors every binding intoe.varson the resolver-verified slot fast path (skip the chain walk whenslot >= 0, defer the name-map mirror to rare name lookups).Runbuildsi.globalwithout pre-sizing fromprog.NumGlobals, growing the slot slice one-at-a-time (O(n^2) for n globals).execIndexAssign/execAppend/execFieldAssignre-fetch the root binding by name though the rootVarExpralready carries(Depth, Slot).lists.reverse/head/tail/slice/concatdeep-copy the whole argument then immediately replace the copied slice - the first copy is pure waste.
Acceptance. A map-heavy build ($m[$k] = $v over N keys) is near-linear, not
quadratic; a call-heavy and a loop-heavy benchmark improve measurably on the
reference machine; every existing test (incl. value_alias_test.go and the map
ordering tests) still passes.
M19.4 - Resource lifecycle and numeric strictness
Done. os.spawn handles are now keyed by a monotonic internal id, not the
OS pid, so a recycled pid can never make a later spawn overwrite an earlier
handle or make os.wait / poll / kill hit the wrong process (pinned by a
distinct-handles test); the reaper also drains the captured buffers into strings
and drops the live *bytes.Buffers so a terminated handle stops pinning them
for the program’s life (idempotent os.wait and poll-after-wait are
preserved - literal delete-on-reap would break both, so the persistent-handle
contract stays and an explicit release stays a possible future add). Numeric
strictness: convert.toInt and math.floor / ceil / round reject NaN,
+/-Inf, and out-of-int64-range floats with positioned errors instead of an
unchecked int64(f) cast, math.abs(MinInt64) errors (its magnitude does not
fit), and the toml decoder makes an integer past int64 a decode error rather
than a lossy-float downgrade (json keeps its deliberate fallback). The
most-negative int literal -9223372036854775808 (and -0x8000000000000000)
now parses to MinInt64 - folded at the unary-minus site with ParseUint +
a 2^63 range check - while the bare magnitude stays a range error. The
uncapped-allocation sinks were already capped ahead of this milestone
(net/fs maxReadBytes/maxHandleRead, compress/archive
maxDecompressed). All fixes carry regression tests; full suite green on both
toolchains.
os.spawnhandle lifecycle.internal/lib/os/exec.gokeys process handles by OS PID and never deletes them: everyprocessState(with buffered stdout / stderr) is retained for the interpreter’s life, and after the reaperWait()s a process the freed PID can be reused, so a lateros.spawnoverwrites the entry andos.wait/poll/killon the old handle hits the wrong process. Key handles by a monotonic internal id (likenet/fs/httpd) and delete on reap.- Numeric strictness.
convert.toIntandmath.floor/ceil/rounddo an uncheckedint64(v.Float), platform-defined garbage for NaN, Inf, and out-of-int64-range values - contradicting themathlibrary’s documented strict stance (mathematically-undefined results error, never yield garbage). Reject NaN / Inf / out-of-range with positioned errors; special-casemath.abs(MinInt64)(currently returns a negative). The toml decoder degrades an integer literal past int64 to a lossy float - TOML 1.0 says integer overflow is an error, so make it one (json keeps its deliberate lossy-float fallback). - Most-negative int literal.
-9223372036854775808(and0x8000000000000000) is a parse error because the magnitude is parsed withParseIntbefore the unary minus applies. Parse magnitudes withParseUintand range-check against 2^63 under a leading minus.
The uncapped-allocation issues (caller-supplied make([]byte, n) in net /
fs reads, and unbounded io.ReadAll of decompressed compress / archive
streams - a zip-bomb sink) are fixed immediately, with fixed sensible
defaults mirroring httpd’s maxBodyBytes, ahead of this milestone.
Acceptance. A recycled-PID scenario signals the right process (or errors)
and the handle table does not grow without bound; convert.toInt(NaN) /
math.floor(1e300) error; math.abs(MinInt64) errors; -9223372036854775808
parses; a too-large toml integer is a decode error.
M19.5 - Module struct identity: keyed by canonical path
Done. Module struct values were tagged only by the imported file’s stem
(moduleStem), so two modules whose files share a basename
(import "a/util.j" as u1; import "b/util.j" as u2;, or the M19.7-era
import "@mplx/benchmark/" / "@claude/benchmark/") produced values with an
identical (namespace, name) identity, and moduleByNS resolved the stem
nondeterministically - a same-named struct from an unrelated module passed the
other’s type check.
The original plan here was to fail loud (reject a stem collision at import
time), but that is far too restrictive for real projects (one import would claim
a basename project-wide) and would make the @scope/package case impossible, so
the design was changed to do it properly: struct identity is keyed by the
module’s canonical (resolved) path, not the basename. Value and
parser.Type gained a ModPath field (the module’s canonical path, empty for
library / user structs) that Value.Equal and MatchesDeclared compare
alongside StructNS; StructNS stays the file stem purely for display, so
error messages and %v still read benchmark.Point. The boundary retag
(retagStructs / retagType) threads the path so a foreign struct that merely
shares a stem is never mis-tagged, and method parameter types are now
stamped (resolveDeclaredTypesOnce) so a func f(s as mod.Struct) param carries
the identity the passed value does. Two imports that resolve to the same file
stay the same type (path-resolved before comparison, so ./x.j and ../y/x.j
collapse); different files are different types. No import errors, no collision to
reject. Pinned by a same-stem-modules-coexist test (each module’s struct
round-trips through its own methods; the two types do not cross-satisfy) plus
distinct-stem / re-import tests; full suite, all 53 overlays, the web
cross-boundary integration, and -race stay green on both toolchains.
Acceptance. Two module files sharing a basename import cleanly under distinct aliases and are distinct types that never cross-satisfy; re-importing the same file is the same type; single-stem programs and the module test suite are unaffected.
M19.6 - .j code coverage
Done. jennifer test --coverage[=text|json] reports statement coverage by
reusing the profiler’s per-position hit data (no second counting path):
loadForTestProg installs a statement profiler before running so hits are
captured from top-level init through every test method, and renderCoverage
intersects those hits with the executable positions statically walked from the
AST (statementPositions, which mirrors execStmt’s per-statement recording so
the sets line up). It is scoped to the tested program’s files - an imported
module that merely ran does not skew it - and reports per-file plus a total; a
module overlay shows the module and test files separately. text (default)
prints percentages and names the never-executed positions; json emits a
parseable report that owns stdout (the human test report moves to stderr, the
profile --format=pprof rule). The plain jennifer test path is unchanged
(loadForTest is now a thin wrapper passing a nil collector). Pinned by
render-logic unit tests and end-to-end CLI tests (partial coverage names the
uncovered lines; json parses on stdout; the plain run emits no coverage), and an
HTML view is left as a later htmlwriter consumer.
Teach jennifer test to report which lines of the code under test actually
ran. The profiler (jennifer profile) already records per-position hit
counts, so the raw signal exists; this milestone surfaces it as coverage: a
per-file and total percentage, the list of never-executed positions, and a
machine-readable form a CI job or an editor can consume. It reuses the
profiler’s instrumentation rather than adding a second counting path, and is
independent of any renderer: a plain-text summary is the baseline output.
Educationally it closes the loop the REPL / linter / profiler / test-runner
set opened - a learner sees not just that tests pass but what the tests
miss.
- Surface.
jennifer test --coverage[=FORMAT]runs the suite with the profiler’s counters live over the tested file(s) and emits a coverage report next to the test report. Defaulttext(per-file and total percent); plus a machine-readable form for tooling. - Reuse, do not duplicate. Coverage is a second consumer of the profiler’s per-position hit data, not a new instrumentation pass.
- No renderer dependency. Text is the baseline; an HTML coverage view
would be a later consumer built on
htmlwriter(M18.2), not part of this milestone.
Acceptance. The coverage report over a file whose tests exercise some
but not all of its methods shows below 100 percent and names the unexecuted
positions; a suite that touches every position shows 100 percent; the
machine-readable form parses; the plain jennifer test path (no
--coverage) is byte-for-byte unchanged.
M19.7 - @scope/package module resolution (vendored packages)
Done. A leading @ is a vendored-deck reference, expanded by one function
(resolveVendor in internal/module): the @ swaps in the vendor root and a
reference not ending in .j gets the package-named entry appended, so
@claude/bitcoin, @claude/bitcoin/, and an explicit @claude/bitcoin/utils.j
all reduce to a plain absolute path - after which resolution, the run-once
cache, and M19.5 path identity are untouched. Because the entry is
<package>.j (named after the deck), moduleStem gives the package name, so
the display namespace and default alias fall out with zero special-casing
(import "@claude/bitcoin/" binds bitcoin.); two same-package decks across
scopes (@a/bitcoin, @b/bitcoin) are distinct types (M19.5) and collide only
on the default alias, resolved with as. The vendor root comes from
module.FindVendorRoot with the sysmoddir-style layering (--vendor flag on
run, else JENNIFER_VENDOR, else the nearest vendor/ above the program;
wired into run / repl / test via SetVendorRoot). Path safety: @ only
at the front, no ./.. segments, and the resolved file must stay inside the
deck directory; a missing vendor root is a guided error, not a crash. The parser
exempts @ deck references from the .j requirement. Pinned by module-package
unit tests (expansion, error cases, vendor-root discovery) and end-to-end CLI
tests (entry / explicit-file / default-alias imports, cross-deck type
distinctness, missing-root error); full suite, all 53 overlays, and both
toolchains stay green. The jvc manager layered over this stays DRAFT#12.
Today the module resolver keys on the import path’s leading token: ./ (or
../) is local (relative to the importing file), / is absolute, and a bare
name resolves through the search path (system module dir, then each -I DIR).
None of those reach a package installed beside the app rather than
system-wide, and a bare import "util.j"; deliberately never consults the
importing file’s own tree, so it cannot address a vendored package and would
collide with a system module of the same stem. A package manager (the planned
jvc, DRAFT#12 in horizon.md) installs packages into a
project-local vendor/SCOPE/PACKAGE/ tree; this milestone teaches the resolver
to address that tree, so a vendored package imports unambiguously and
independently of the system search path.
- A fourth leading-token kind, expanded by one function.
import "@scope/package/" [as alias];addresses a deck under the vendor root. Both ends are pure path expansion, done once inexpandModule(): a leading@swaps in the vendor root (@is the<vendorRoot>/shortcut), and a reference that does not end in.jgets the deck’s entry file appended, so@claude/bitcoin,@claude/bitcoin/, and an explicit@claude/bitcoin/utils.jall reduce to a plain absolute path. After that expansion every downstream step (resolution, run-once cache, M19.5 path identity) is unchanged - it is just a path import, and the vendor variable only affects this expansion. It stays an ordinary string-literal path, so there is no new grammar (the inline version selectors@pkg=1.2.3/@pkg#commitarejvc’s concern, DRAFT#12). - Package-named entry, so the namespace is free. A deck’s entry file is
<package>.j(vendor/claude/bitcoin/bitcoin.j), named after the deck directory. Because a module’s display namespace and default alias both come frommoduleStem(the basename), that convention makes them fall out with zero special-casing:import "@claude/bitcoin/"binds thebitcoin.namespace anddef x as bitcoin.Wallettype-checks, all from the existing stem logic. Without an alias the default namespace is thus the package name (never the uselessmaina fixed entry name would give); two decks with the same package name (@a/bitcoin,@b/bitcoin) default to the same alias, collide, and take an explicitas(the two-util.jrule) - typical use is aliased anyway. - Vendor-root discovery. Walk up from the entry program’s directory to the
nearest
vendor/(thenode_modules/ Composer convention), overridable with--vendor DIRandJENNIFER_VENDOR- the same override layering as--sysmoddir/JENNIFER_SYSMODDIR. A missing vendor root, or a missing@scope/package, is a positioned error naming the resolved path and pointing at the installer. - Identity already handled by M19.5. Because struct identity is keyed by the
resolved canonical path
(M19.5), two decks
that share a stem (
@a/util,@b/util) resolve to different vendor paths and are already distinct types - this resolver just has to produce the right canonical path per scope; no extra identity work. - Path safety.
@is legal only as the very first character of the path; a..segment may not escape the package root; the resolved path must stay insidevendor/scope/package/.
The full jvc package manager (registry, deck.toml manifest, lockfile,
version constraint solving, fetching) stays a beyond-1.0.0 draft (DRAFT#12);
this milestone lands only the interpreter-side resolution jvc builds on, so a
vendored tree populated by hand (or by an early jvc) imports today.
Acceptance. import "@acme/widgets/" as w; in an app with
vendor/acme/widgets/widgets.j resolves and runs (the trailing / expands to
the package-named entry); an explicit @acme/widgets/util.j also resolves;
without an alias the namespace defaults to widgets; the vendor root is found by
the upward walk and is overridable via flag / env; a missing package is a
positioned error; ./, /, and bare-name imports are byte-for-byte unchanged;
@a/bitcoin and @b/bitcoin structs do not cross-satisfy each other’s type
checks; a ..-escape path is rejected.
M19.8 - Relocation: jennifer-language org + vanity module path
A one-time project relocation, no language or interpreter behavior change. The
repository moves from the personal github.com/mplx/jennifer-lang to a
jennifer-language GitHub org, and - separately - the Go module path moves off
GitHub to a vanity import path (jennifer-lang.dev/jennifer) served by a
go-import meta page, so the module identity no longer depends on where the
code is hosted. Names are settled: the domain jennifer-lang.dev is registered,
and the org is jennifer-language because jennifer-lang and jenniferlang
were already taken on GitHub. The domain (-lang) and org (-language) spelling
differing is deliberate and invisible in use - the canonical module path is the
domain, and the org is only the git host the meta page points at. Purely
mechanical, but it touches nearly every file (112 .go
imports, ~60 docs, CI, packaging), so it gets a milestone to keep the sweep
complete and reviewable, and it lands before the first post-org release so
downstream references (AUR, doc links) migrate exactly once. Two distinct
targets - keep them separate:
- Go module path -> the vanity domain.
go.modbecomesmodule jennifer-lang.dev/jennifer; every internal import (github.com/mplx/jennifer-lang/... -> jennifer-lang.dev/jennifer/..., 112.gofiles) is rewritten. A one-filego-importmeta page atjennifer-lang.dev/jennifermaps the vanity path to the org repo (<meta name="go-import" content="jennifer-lang.dev/jennifer git https://github.com/jennifer-language/jennifer">, plus ago-sourcetag for pkg.go.dev deep links), served from the same site that hosts the mdBook docs. The path is now host-independent: a future repo move never touchesgo.modagain. - Human-facing URLs -> the org repo. Clone URLs, issue links, and every
github.com/mplx/jennifer-lang/blob/main/...deep link move togithub.com/jennifer-language/jennifer(flagship repo namedjennifer, not doubled - therust-lang/rustshape). These point at GitHub, not the vanity domain (the.devhost only serves thego-importpage and the docs). GitHub’s transfer redirect keeps old links working, but every canonical URL in-tree is updated rather than left to the redirect. Sibling repos (jvc, the deck registry) are created empty under the org as their own milestones land, not here. - Metadata / CI / packaging sweep.
README.mdbadges and links,docs/**(installing, tooling, user-guide, technical),CLAUDE.md,JENNIFER.md,modules/README.md,book.toml, the one example that hardcodes the URL (examples/modules/barcode_demo.j), the workflows (.github/workflows/{test,docs,release}.yml), and the packaging manifests (packaging/arch/PKGBUILD-{bin,git},packaging/arch/publish-{bin,git}.sh,packaging/debian/copyright,scripts/build-deb.sh) all move to the new host.grep -rn 'mplx/jennifer-lang'comes back empty at the end. - Jennifer deck scope. The first-party package scope in docs / examples
flips from the placeholder
@mplx/...to@jennifer/...(DRAFT#12 and the M19.7 examples), matching the org and the registry identity the vanity domain anchors. Doc-only untiljvcand the registry exist.
Acceptance. go get jennifer-lang.dev/jennifer/cmd/jennifer resolves
through the vanity meta page to the org repo; go build ./... and go test ./... pass under the new module path; make build (both toolchains) still
produces jennifer / jennifer-tiny; grep -rn 'mplx/jennifer-lang' is empty;
the old GitHub URL redirects to the org; the AUR package builds from the new
source; pkg.go.dev serves the module under jennifer-lang.dev/jennifer.
M20 - system libraries
Go system libraries: cryptographic primitives, plus formats too heavy
or too reflect-bound for a Jennifer-coded .j module (the json pattern,
M16.9). Members below; more land as M20.x as needs arise.
M20.1 - crypto
Symmetric and asymmetric primitives, key derivation, crypto-grade random. System library; TinyGo-safe primitives only. Hashes already shipped in M15.6.
- Swap
uuid’s random source.uuid(M16.10) draws its v4 / v7 randomness frommath’s shared non-crypto RNG (seedable, predictable - documented). When crypto-grade random lands here, repointuuidlib.randByteat it souuid.v4is unguessable; the change is one function, no surface change. Until thenuuidmust not be used for security tokens. - Message authentication (HMAC) already shipped as
hash.hmac. HMAC is a hash construction, so it lives in thehashlibrary (RFC 2104, Gocrypto/hmac, TinyGo-clean) rather than here - it unblocks request signing, webhook verification, JWT HS256, and TOTP without a crypto library. What can still land here is a constant-timecrypto.hmacEqualfor MAC comparison (verification today recomputes and compares the full digest). The KDFs below build on HMAC (PBKDF2 is iterated HMAC; HKDF and SASL SCRAM are HMAC-based). - Key derivation (stdlib, no dependency).
HKDF- derive keys from a high-entropy secret - andPBKDF2-HMAC-SHA256- derive a key from a password (salt + iteration count). Both come from the Go standard library (crypto/hkdf,crypto/pbkdf2, stdlib since Go 1.24), so they add no dependency and stay TinyGo-clean. Shape:crypto.hkdf(...)/crypto.pbkdf2(password, salt, iterations, keyLen) -> bytes. - Password hashing is out of scope here. Argon2id (and bcrypt /
scrypt) moved to the Long-horizon list: they need the
x/cryptodependency and their ownhashPassword/verifyPasswordsurface.
M20.2 - xml
Hand-rolled like json (Go’s encoding/xml is reflect-heavy, so
TinyGo-hostile). A genuinely complex tree - attributes + ordered,
possibly-duplicated children + mixed text + namespaces + entities - whose
byte-level parsing is too slow in .j. Also the natural mirror target for
the json.Value read / write vocabulary
(M16.16): the same opaque-handle plus path-addressed
accessor shape (an XPath-style path dialect in place of JSON Pointer).
M20.3 - yaml
A system library because full YAML - anchors / aliases, flow and block
styles, implicit typing, multi-document streams - is impractical in .j
and has no Go stdlib. Unlike xml, that means a Go dependency (e.g.
gopkg.in/yaml.v3): the one place a config parser earns one, since a
hand-rolled full YAML is a project of its own. Verify TinyGo-cleanliness
of the dependency, and fall back to a documented subset if it won’t build
there.
M20.4 - i18n
Message catalogs and locale-aware translation. A system library, not a
.j module, for two independent reasons: it needs global mutable state
(the current locale plus loaded catalogs), which a declarations-only module
cannot hold; and it needs performance - Jennifer’s map is a linear-scan
[]MapEntry, so a large catalog looked up per call would be O(n), whereas the
library holds catalogs in a Go map[string]string (O(1)). The
map of string to string a caller passes is fine as the load interface (a
one-time ingest); the per-lookup scan is what the library avoids.
Surface: i18n.load(lang, catalog) (a map of string to string, built from
json today or yaml here at M20.3, or a literal), i18n.setLocale(lang) /
i18n.locale(), i18n.tr(key) (translate in the current locale; fallback
locale -> default lang -> the key itself, so a missing key is visible), and
i18n.tr(key, params) for named interpolation ("Hello, {name}").
Pluralization (CLDR per-language plural rules) and an i18n.loadFile(path)
convenience are follow-ons.
No gettext-style _(): _ is not a valid Jennifer method name (letters-only;
_ is reserved for constant-name separators), and a bare tr() builtin does
not clear the len promotion bar (translation is not useful to nearly every
program). Ambient-global _() is also exactly what stances 2 (explicit) and 7
(namespaced, no globals) rule out - so the call is i18n.tr("key") (or
use i18n as t; t.tr("key")). Extending printf for translation is rejected
(see technical/rejected.md): translation is content
substitution from stateful external data, not presentation of the value in
hand. Locale-aware value formatting (number / date grouping) is a separate,
open printf question.
M20.5 - term
A term system library exposing the terminal host capabilities a TUI needs
and pure .j cannot reach: raw mode (makeRaw / restore - unbuffered,
no-echo input), terminal size (size -> rows, cols), and raw single-key
reads from stdin. It reuses golang.org/x/term - already a repository
dependency scoped to the REPL’s line editor (cmd/jennifer/lineedit.go) - so
it largely exposes a capability the interpreter already exercises. Build-tag
split like net / os: a friendly-error stub on jennifer-tiny (embedded /
minimal targets may have no controlling TTY). This is the enabler for
interactive TUIs; the pure-ANSI screen control, key decoding, and rendering
sit in the M21.1 screen / tui module on top.
Output-only TUIs (dashboards, progress bars) need neither this library nor raw
mode - just ansi + os.isTerminal.
M20.6 - hardware buses (serial / spi / iic)
Device-bus I/O libraries for embedded / single-board-computer hosts - the
syscall-backed siblings of the sysfs-backed gpio module
(M18.11). Each needs ioctl / termios, which .j and
plain fs cannot reach, so they are Go system libraries, not modules:
serial- open a serial port (/dev/ttyUSB0,/dev/ttyAMA0), configure it (baud rate, data bits, parity, stop bits viatermios), thenread/write/close. Setting the baud rate is atermiosioctl, unreachable fromfs- which is what forces a library.spi- open a SPI device (/dev/spidev0.0), set mode / speed, andtransfer(bytes) -> bytes(full-duplex), via theSPI_IOC_MESSAGEioctl.iic- the I2C bus (/dev/i2c-1): select a slave address andread/writeregister bytes via theI2C_SLAVEioctl. Namediic(Inter-IC) rather thani2cbecause a library namespace is letters-only (no digit, likebucketnots3); candidatesiic/twi/wire, settled at build time.
Build-tag split like net / os: the real implementation on Linux (the
supported platform), a friendly-error stub elsewhere. Default binary; a
jennifer-tiny rebuilt for a specific board could include them (TinyGo’s
machine package is a different, microcontroller-level API - these target the
Linux /dev + ioctl interface). Together with gpio they complete the SBC
I/O story.
M20.7 - sql (MySQL / MariaDB + PostgreSQL)
A relational-database client library over Go’s database/sql, shipping the
two client-server engines: MySQL / MariaDB (go-sql-driver/mysql) and
PostgreSQL (jackc/pgx), both pure-Go drivers (no cgo, so cross-compile
and the best-effort macOS / Windows artifacts stay clean). SQLite - the one
embedded engine - is deliberately not here; it needs a multi-MB
dependency and cannot build under TinyGo, so it stays a build-tag opt-in
parked in horizon (the jennifer-full variant).
Why a Go library and not a .j module over net. MySQL and Postgres are
open TCP wire protocols, so a client is writable in pure Jennifer - the same
shape as redis / memcache / imap, and the auth crypto it needs (SHA-1,
SHA-256, hash.hmac, PBKDF2 as iterated HMAC) already ships. The deciding
factor is not performance: a database client is latency-bound (network
round-trip + server execution dominate; client-side decode only becomes the
cost center when streaming 10^5+ rows, a bulk workload Jennifer is not the
right tool for regardless of driver), and the COW shared-marker protocol
already makes materializing a large result set amortized O(N). The deciding
factor is correctness maturity: the mature drivers have absorbed a decade
of protocol long-tail - every auth plugin (MySQL caching_sha2_password
full-auth / the RSA path, Postgres SCRAM-SHA-256), charset handling, NULL
semantics, multi-result-sets, server-version quirks - that a hand-rolled .j
client would re-derive one edge case at a time. For databases users depend on
daily, that maturity is worth the dependency. Going Go also makes the auth
crypto the driver’s problem, not the language’s, so this needs nothing from
M20.1 crypto.
The deliberate dependency break. These are the first heavyweight
dependencies in the library layer - a conscious exception to the
dependency-free discipline CLAUDE.md states for the library layer (the
only third-party dependency today is CLI-scoped golang.org/x/term). Both
drivers are pure-Go, but they are real dependency trees. The exception gets a
reasoning record in technical/design-decisions.md
when it lands - the sanctioned home for a feature that ships despite appearing
to cut against project doctrine - justified as above, not slid into.
Build / TinyGo. Build-tag split exactly like net / httpd:
sqllib_std.go (//go:build !tinygo) imports database/sql + drivers;
sqllib_tiny.go (//go:build tinygo) registers use sql; and returns a
friendly positioned “not available on this build” error. TinyGo never
compiles the driver imports, so the language stays TinyGo-clean and the
interpreter core is untouched. On stock jennifer-tiny the engines are
unavailable anyway (no net driver), consistent with every net-backed module.
Surface. Integer-handle-into-a-registry like fs / net:
sql.open(driver, dsn) -> Connection, sql.query(conn, sql, params...) -> Rows, sql.exec(conn, sql, params...) -> Result (affected-rows /
last-insert-id), prepared statements, sql.begin / commit / rollback
transactions, sql.close. Values bind only through placeholders
(injection safety) - database/sql abstracts the per-driver spelling (?
for MySQL, $1 for Postgres). The result-row shape is an opaque sql.Row
KindObject mirroring json.Value (foreshadowed in interpreter note 18),
walked by accessors; a typed-struct path waits on the deferred map-to-struct
conversion. A .j postgres.j module (Postgres has the clean protocol, no
RSA gap) remains a possible later optional dependency-free alternative and
language stress-test - not the primary path.
Builds on. The M21.5 orm module is layered on this
library (its hard prerequisite) and inherits the sql.Row caveat above - its
typed-row ergonomics wait on the same deferred map-to-struct conversion, so it
graduates in stages.
M21 - general backlog (catch-all)
The general holding area for milestones that fit no other bucket - not a Jennifer-coded module (M18), not interpreter / tooling work (M19), not a Go system library (M20), and not a beyond-1.0.0 idea (embedding, WASM, and the rest live in the horizon collection). It is the top-level counterpart to M19’s tooling bucket: anything worth recording that has no natural home lands here as a numbered sub-entry, and graduates out into its own bucket once a cluster grows enough to deserve one.
M21.1 - screen / tui module
Jennifer’s terminal-UI answer - a .j module for terminal user interfaces,
since there is no GUI medium-term, so the terminal is the interactive surface.
Layered so the output-only subset ships with no host dependency:
- Stage 1 (no prerequisite). Pure-ANSI screen control over
ansi: cursor movement, clear, alternate-screen buffer, hide / show cursor, box-drawing, and a screen buffer + render / diff loop. Enables output-only TUIs - live dashboards, progress bars, spinners, self-updating tables (therich-style subset). Pure strings; TinyGo-clean. - Stage 2 (needs M20.5). A key-event decoder (parse the raw
byte stream -
ESC [ A-> Up, and so on) plus an event loop overterm’s raw mode / size / key reads. Enables interactive TUIs - menus, forms, key navigation (thecurses/bubbleteasubset).
Positioned as an explicit terminal UI, not a GUI framework. Named screen
or tui (settled at build time). Parked in the general backlog for now; it can
graduate into the M18 module track when built.
M21.2 - feed module (RSS + Atom)
A feed module for web syndication feeds: build and parse both RSS 2.0
and Atom 1.0. One module, format selected on build and detected on parse
(design stance 1 - not separate rss / atom modules); a feed is a value-
semantic Feed (title, link, updated, entries) of Entry (title, link, id,
published / updated, summary, content). It composes across the stack: http
fetches a feed, feed parses it, time handles the RFC 3339 / RFC 822 dates -
a feed reader, podcast client, news aggregator, or changelog-to-feed generator
in a few lines.
Parked here because it is gated on M20.2 xml: RSS / Atom
parsing needs a real XML parser (entities, CDATA, and Atom’s XML namespaces),
which is exactly why xml is a system library rather than a hand-rolled .j
scanner - so feed parsing rides it. Feed building (emitting escaped XML)
could predate xml, but build and parse ship together on the same layer. Also
uses http (fetch) and time (dates); pure .j otherwise. It graduates into
the M18 module track once xml lands. Discipline as usual: a
modules/feed_test.j overlay (build / parse round-trips and date handling for
both formats as pure helpers; a networked fetch-and-parse against an in-process
server in a Go test), docs/modules/feed.md, a catalog row, a SUMMARY.md
entry, a modules/README.md entry, a JENNIFER.md bullet, and a runnable
examples/modules/feed_demo.j.
M21.3 - jwt module (JSON Web Tokens)
A JWT (RFC 7519) module. The HMAC algorithms (HS256 / HS384 / HS512) need only
the shipped hash.hmac and could stand alone, but the module targets the full
common surface - including the asymmetric RS256 / ES256 that OAuth / OIDC
rely on - so it is parked here gated on M20.1 crypto for
the public-key signing / verification, and graduates into the M18 module track
when crypto lands. Surface: jwt.sign(claims, key, alg), jwt.verify(token, key) -> claims (checks the signature and the exp / nbf time claims), and
jwt.decode(token) -> claims (read without verifying). Over hash.hmac (+
crypto for RS / ES), encoding (base64url), json, and time. jwt_auth
is not a separate module: it is this module used as a web.before middleware
that pulls the bearer token from the Authorization header, jwt.verifys it,
and rejects on failure (func requireJwt(ctx) { ...; return true; }) - shipped
as a snippet in the demo / docs, not its own surface. Discipline as usual: a
modules/jwt_test.j overlay (sign / verify round-trips and tampered-token
rejection), docs, catalog, and demo.
M21.4 - acme module (Let’s Encrypt / ACME)
An ACME (RFC 8555) client - obtain and renew TLS certificates from Let’s Encrypt
and compatible CAs: account registration, an order plus HTTP-01 / DNS-01
challenge, CSR submission, and certificate download, over http + json. Parked
here gated on M20.1 crypto: ACME requests are JWS-signed
with an account key (RS256 / ES256) and the flow needs CSR generation - both
asymmetric-crypto operations. Composes with web / httpd to serve the HTTP-01
challenge. Graduates into the M18 module track when crypto lands. Needs the
default binary. Discipline as usual.
M21.5 - orm module
A relational mapper layered over the M20.7 sql
library (its hard prerequisite - no sql, no orm), and a good stress-test
of how far the module system stretches. Jennifer’s semantics dictate the shape,
and it is Data Mapper, not Active Record: structs are value-semantic and
carry no methods, and a module is declarations-only with no mutable state, so a
row object cannot know how to save() / delete() itself and there is no place
to hold identity-map or dirty-tracking state. The pattern is therefore a
repository / table-gateway - the caller passes a record and a schema to
module functions: orm.insert(conn, schema, record), orm.find(conn, schema, id) -> record, orm.update / orm.delete, orm.all(conn, query).
Three constraints shape the surface:
- Explicit schema descriptor (no reflection).
.jcannot introspect a struct’s fields, so the caller declares the mapping once - anorm.Schema(table name, column-to-field list, primary key, per-column type) built with a small constructor. This is the module’s central object; everything keys off it. - Non-mutating functional query builder, mirroring the
json.Valuewrite surface (set/insert/appendreturning fresh handles) rather than method chaining (values have no methods):orm.where(orm.from(schema), "age", ">", 18)returns a neworm.Query, composed functionally throughorm.where/orm.orderBy/orm.limit/orm.join, then rendered to parameterized SQL - values bind only through placeholders (injection safety inherited fromsql). A small dialect layer (placeholder spelling?vs$1, identifier quoting,LIMIT/OFFSET,RETURNING) is a backend selector on one module, not parallel modules (stance 1 / the one-module rule). - Row-to-struct mapping is partly gated.
sql.queryyields an opaquesql.Row; turning it into a typed struct wants the deferred explicit map-to-struct conversion (horizon). Soormships in two steps: first amap of string to V(or by-hand field extraction via the schema) row form that needs nothing new, then the typed-struct return once map-to-struct lands.
Transactions come straight from sql.begin / commit / rollback. Kept out of
v1: relations beyond a plain join (has-many / belongs-to eager loading wants
object identity and lazy proxies Jennifer does not have), and migrations (a thin
orm.createTable(schema) DDL emitter is a plausible follow-on, full migration
tooling is its own module). Needs the default binary. Discipline as usual: a
modules/orm_test.j overlay, docs, catalog, and demo - with the overlay split
so the query-builder-to-SQL surface (pure string generation) is covered 100%
offline, and live CRUD sits behind a DB-service-gated integration test rather
than the unit overlay.
Requirements for 1.0.0 stable
The core CI + release + packaging items that used to live here were promoted into M15.8 (the last step before Phase C). What stays here are the distribution requirements for a stable 1.0.0 that aren’t themselves milestones - they can land any time and don’t block any feature milestone:
- Cross-build for macOS / Windows. Waits on the platform-portability work in the horizon ideas; ships as soon as that lands.
- Real apt repository (replacing the “GitHub Release
artifact” install of the M15.8
.deb) if user demand warrants the maintenance.
The extra Linux / macOS distribution formats (Homebrew, Snap, Nix, Flatpak, AppImage, …) are not requirements; they live in the horizon idea collection and ship when there’s user demand and a maintainer willing to keep one green.
Long horizon
Ideas for development beyond 1.0.0 - embedding, a WASM runtime, specialised-domain libraries, and a grab-bag of smaller possibilities - live in their own collection, kept out of the near-term plan so this file stays focused on the road to 1.0.0. See the beyond-1.0.0 idea collection.