Developing Kipferl

This guide is for changing Kipferl itself. To develop a Python application, start with project commands and kipferl dev. The workflow below changes the source checkout. Application developers can install v0.7.2 to use the new project and package commands without installing the contributor tools. Homebrew tracks stable releases; contributor tooling remains in the repository.

Reproduce the setup

You need Git, mise 2026.9.1 or newer, and a native C compiler and linker. On macOS, install the Xcode Command Line Tools; on Linux, install your system's C build tools. PocketPy's vendored C source is compiled as part of the Rust build.

git clone https://github.com/niklas-heer/kipferl.git
cd kipferl
mise trust
mise install --locked rust python node bun cargo:bacon aqua:nextest-rs/nextest/cargo-nextest watchexec
mise run setup

Review the repository's task definitions before trusting them. mise run activates the pinned tools, so shell activation is optional. Setup verifies tools and the compiler, fetches locked Cargo dependencies, installs website dependencies with Bun's frozen lockfile, and builds the release workspace. It can be rerun after pulling changes.

mise.toml, mise.lock, and rust-toolchain.toml are authoritative. Native SDKs, system libraries, and linkers remain host prerequisites; the pins do not make different operating systems identical. Run mise run doctor to diagnose tool drift. Discover the available tasks with mise tasks ls.

Choose a feedback loop

mise run bacon

Bacon starts with compiler checks. Press c for Clippy, t for full Rust tests, r for core-profile Clippy, Shift-R for core tests, or d for doctests. It exports navigable locations to target/bacon-locations for editor integrations. Rust Analyzer and standard-library sources are part of the pinned toolchain; launch your editor with mise exec -- <editor> . if it does not inherit them.

mise run watch

The watcher queues a check–test–build sequence when Rust, vendored C, or relevant configuration changes. It does not launch an application or a release helper. Use kipferl dev separately when testing a Python app's behavior.

For one targeted Rust integration suite, nextest accepts package and test filters:

mise exec -- cargo nextest run --locked -p kipferl-cli --test diagnostics

Test retries are disabled. Slow tests are reported, then terminated after the configured limit. Nextest also fails tests whose descendants retain inherited output streams. These checks complement the CI memory sanitizers; they do not replace them. Keep doctests in mise run test-doc, because nextest does not execute them.

Run the contribution gates

The strict audit runs pedantic, nursery, and the selected restrictions on panics, unwraps, indexing, arithmetic, casts, and process exit. Its report at target/lint-audit/report.md separates unresolved diagnostics from explicit allow and expect declarations. exceptions.json records each declaration's location and reason, including conditional declarations.

Fix the cause of a diagnostic. When a specific operation relies on a proven invariant, a narrowly scoped #[expect(clippy::specific_lint, reason = "...")] can document it. Blanket Clippy groups and allow(warnings) are rejected. Do not add allowances to turn an unexamined finding into a passing check. Test-fixture setup assertions have their own documented expectations.

CI runs the same source gates, retains JUnit and lint-audit reports, exercises four release targets, and checks the PocketPy FFI with Linux sanitizers. A local macOS run does not establish the Linux sanitizer or cross-target results.

Measure before changing performance

mise run bench writes Criterion reports beneath target/criterion/. Compare the same build profile and machine with an idle system; a changed benchmark is evidence to investigate, not a universal speed claim. Loader microbenchmarks and the end-to-end scripts in benchmarks/ measure different costs.

mise run seek optionally installs the pinned cargo-seek task tool for interactive crate, feature, and MSRV inspection. It does not add dependencies to the application. Keep production dependencies feature-minimal and update lockfiles deliberately.

Change runtime code and embedded assets deliberately

mise run build produces target/release/kipferl, pocketpy-kipferl, and kipferl-loader. To exercise newly compiled native runtime behavior directly:

mise run runtime 'import json; print(json.dumps({"ok": True}))'

The public CLI embeds prebuilt, target-specific components from crates/kipferl-cli/assets/. Rebuilding Rust source alone does not refresh those asset files. Test runtime changes against the newly built runtime, then follow the repository's release process to refresh and verify components before claiming that a distributed CLI contains them. The release workflow rebuilds components for all four targets before assembling release CLIs.

After changing the canonical .pyi stubs, run mise run stubs and mise run stubs-check. Regenerate FFI bindings with mise run bindings when needed; it additionally requires bindgen and libclang. Never edit generated bindings or the generated stub manifest independently of their sources.

Extend package compatibility evidence

The CLI combines reviewed compatibility/packages/catalog.json records with verified syntax blockers from compatibility/packages/popularity-catalog.json. Its lookup key combines the normalized distribution name, exact version, wheel SHA-256, runtime binary SHA-256, and operating-system/CPU target. A rebuilt runtime with the same release version still needs its own evidence. A tested package also does not waive the checks on its transitive dependencies.

Validate the catalog and its reviewed smoke-hook hashes offline:

mise exec -- python3 scripts/package_catalog.py --check
mise exec -- python3 -m unittest discover -s scripts -p 'test_package_catalog.py'

Edit compatibility/packages/candidates.json to propose a pinned package and add focused behavior checks under compatibility/packages/smoke/ when useful. Record exactly which APIs and error paths the hook exercises. The initial tzdata==2025.2 example verifies version constants and four TZif resource headers; it does not claim timezone conversions work.

To gather compilation evidence for both a fresh macOS ARM64 runtime and the binary actually embedded by the CLI:

mise exec -- python3 scripts/package_catalog.py \
  --runtime target/release/pocketpy-kipferl \
  --runtime crates/kipferl-cli/assets/pocketpy-kipferl-macos-aarch64

This refresh downloads wheels, verifies their hashes, and compiles source without importing or executing package code. It replaces the catalog with the supplied runtimes' results. Keep every runtime that should retain evidence and review the resulting diff; compilation alone leaves a package unverified.

After reviewing the exact wheel code and its checked-in hook, adding --execute-reviewed runs behavior checks. That mode currently requires macOS sandbox-exec, limits execution time, and restricts network, writes, and home directory reads. It is a constrained developer runner, not a complete boundary for arbitrary hostile code. Package installation never runs these hooks. Other target evidence must come from checks on that target.

Read the catalog contributor reference for the schema, execution restrictions, and refresh procedure. Keep source compilation failures, behavior failures, and missing evidence distinct; never label a new artifact or runtime tested by copying an older record.

Investigate popular-package findings

The popular package audit keeps its ranking snapshot and artifact findings separate from the reviewed installation catalog. Its canonical JSON, CSV, and Markdown report live under compatibility/packages/. The website reads the JSON during its static build, so changing the audit data requires a website rebuild. The CLI also embeds its report at compile time; rebuild the CLI after regenerating either audit or catalog evidence.

Use the source and runtime hashes to reproduce a finding. Download counts include automation, and one selected release is not evidence about every version of a project. Compilation-only results remain unverified until focused behavior checks establish a narrower tested scope. Follow the contributor procedure for ranking refreshes and bounded audit runs.

Work on the website

mise run website-dev
mise run website-check
mise run recipes

Content lives in website/content/docs/; navigation is controlled by meta.json. The landing page and historical engineering articles live under website/src/app/. Keep historical release measurements dated, and distinguish current source behavior from published artifacts. Recipe code blocks are checked against examples/recipes/; edit both together when changing a recipe.

Release artifact verification

Stable v0.7.0 passed validation on all four native targets, with fresh reviewed package evidence against each final runtime binary. Candidate hashes and catalog approvals were not reused for different binaries. The release pipeline checks component checksums and exact versions before embedding them, then tests package installation, locked offline restoration, and standalone execution after removing the project and caches. Published package reports state the scope and isolation used for each target. macOS reviewed hooks use sandbox-exec; Linux release hooks require an explicit disposable-CI opt-in and GITHUB_ACTIONS=true, which is a runner trust decision rather than an OS sandbox. Only the pinned reviewed tzdata hook runs. Local Linux execution has no silent fallback.

Starting with 0.7.2, each native release job reruns the pinned 1,000-package source audit against its final runtime. The pipeline reuses release metadata pins, never old compilation results. It publishes popularity-audit-<target>.json, its CSV, and popularity-catalog-<target>.json with checksums. Before building a CLI, verification requires complete coverage, matching runtime and target identities, matching ranking and artifact pins, and consistent generated exports. The CLI embeds its own platform's report and exact compiler blockers. Compilation success never becomes a behavioral approval; dependency closures remain unverified.

Contributors can dispatch the Release workflow on a branch to run these gates without publishing. Only a matching version tag publishes a release, and release candidates leave the stable Homebrew formula and latest stable release unchanged. Publishing a stable release updates that channel and its Homebrew formula. See the release preparation reference.