Caleta: An Evidence-Oriented Trust Layer for Reproducible Local Large Language Model Inference
Caleta is an Apache-2.0 process supervisor and evidence layer around llama.cpp, not a new inference engine. It makes explicit which bytes ran, with which configuration, which interfaces were exposed and how things were measured, without keeping prompts. A non-peer-reviewed technical preprint.
Non-peer-reviewed technical preprint
The measurements are descriptive and come from a single configuration (one Windows CPU machine). They do not show superiority over other engines, output quality, energy efficiency or performance on other hardware.
Summary
Running a model locally does not, by itself, answer several operational questions: which bytes were executed, which backend and options were active, which interfaces were exposed, what was measured and which runs failed. Caleta handles that layer around the engine: it wraps llama.cpp's llama-server in a small trust boundary and emits versioned evidence.
The work separates integrity (a matching SHA-256) from authenticated provenance (who published that value), and privacy controls from operating-system isolation. It only reports tests that were actually run.
Abstract
Local large language model (LLM) inference is often described as private, verified, or reproducible without separating the technical claims behind those terms. This paper presents Caleta, an Apache-2.0 process supervisor and evidence layer around llama.cpp rather than a new inference engine. Caleta combines versioned manifests, exact-size and SHA-256 verification, controlled backend construction, a restricted OpenAI-compatible proxy, prompt-free execution records, benchmark reports, and a small conformance suite. Its contribution is the composition of these mechanisms into explicit contracts that distinguish integrity from authenticated provenance and privacy controls from operating-system isolation. We evaluate the implementation through static checks, fifteen unit tests, deterministic end-to-end tests on Ubuntu, macOS, and Windows, and a pinned real-model experiment on Windows CPU using Qwen2.5-0.5B-Instruct Q4_K_M and llama.cpp b11981. The real experiment validates artifact verification, startup, model listing, non-streaming and server-sent-event chat, and conformance. Three independent benchmark invocations produced fifteen successful warm requests with no failures. In this single configuration, median time to first token was 18.066 ms and median decode throughput was 54.394 tokens/s. These descriptive measurements do not establish cross-engine superiority, output quality, energy efficiency, or hardware-general performance. Raw JSON, commands, and environment metadata are published. Remaining challenges include signed manifests, time-of-check/time-of-use replacement, operating-system containment, proxy-overhead measurement, and broader experimental matrices.
Keywords: local inference; large language models; reproducibility; software supply chain; privacy engineering; benchmarking; provenance; llama.cpp.
How it works
- Verify before running: the exact size and SHA-256 of every artifact against a versioned manifest; if anything is missing or does not match, it does not start.
- Supervise the backend on loopback and let only health, model listing and chat completions through a restricted OpenAI-compatible proxy, with a random internal credential per launch.
- Record execution without prompts: configuration and measurements are kept, request and response bodies are not; benchmarks store the prompt's SHA-256 digest and length.
- Measure with method: cold start is separated from warm requests, raw failures are kept and an unavailable measurement is never turned into a zero.
Results
Beyond static checks and fifteen unit tests, the deterministic end-to-end tests passed on Ubuntu, macOS and Windows. The real-model experiment used Qwen2.5-0.5B-Instruct Q4_K_M and llama.cpp b11981 on an Intel i7-7700K CPU with four threads:
| Metric | Observed value | Scope |
|---|---|---|
| Independent cold starts | 3 | One per benchmark invocation |
| Measured warm requests | 15 | Five per invocation; zero failures |
| Cold start, median | 835.371 ms | Range 660.720–977.129 ms |
| Time to first token, median | 18.066 ms | Range 14.574–28.004 ms |
| Decode throughput, median | 54.394 tokens/s | Range 49.028–68.138 tokens/s |
Limits
- A single real configuration (Windows CPU): no real-model run on macOS or Linux.
- Manifests are unsigned and execution records are not attestations; time-of-check/time-of-use replacement remains open.
- There is no operating-system containment: the backend's offline mode is not a firewall, and loopback does not isolate from other local processes.
- It does not measure proxy overhead, energy, memory, thermal behavior or output quality.
Read and cite
The official link is the Zenodo DOI, 10.5281/zenodo.23763881, where the PDF and the experiment artifacts (manifest, commands, environment and raw JSON reports) can be viewed and downloaded. The earlier version remains archived at 10.5281/zenodo.23761755. The reference code is at github.com/BiVelio/caleta.
How to cite it:
Fisa, Joan-Marc (2026). Caleta: An Evidence-Oriented Trust Layer for Reproducible Local Large Language Model Inference. Technical preprint, BiVelio Inc. CC BY 4.0. https://doi.org/10.5281/zenodo.23763881
Manuscript © 2026 BiVelio Inc, licensed CC BY 4.0: attribution must identify Joan-Marc Fisa as author and BiVelio Inc as copyright holder. The reference software, Caleta v0.1.0, is licensed Apache-2.0.
- #preprint
- #local-inference
- #reproducibility
- #llm
- #provenance