Ticket graphs

Every ticket as the root of its provenance graph — goal, patch, approval, runs and (as the ticket-class build lands) mandate, manifestations, guarantees, journey, artifacts and children. Rendered through the <<<TicketGraphList>>> tag.

← all ticket-graphs

REQUESTED#497 omni-git

omni-git: stream blob put/get/verify + OMNI_CHUNK_BYTES RAM cap — safe large-file ingest (no whole-file-in-RAM)

goal

PROBLEM omni-git is not safe for very large files today. The store STRUCTURE is fine (flat content-addressed store, one file per blob at objects/<sha[:2]>/<sha[2:]> — NOT real git, so no packfile/delta/history bloat). The real failure mode is MEMORY: the blob paths read whole files into RAM. Evidence (omni/store.py): * put_file(): return self.put_blob(fh.read()) # entire file -> RAM * put_blob(data): sha = sha256_bytes(data); ... fh.write(data) # hashes+writes whole buffer * _blob_is_intact(): sha256_bytes(fh.read()) # re-reads whole blob to verify (every dedup hit) * get_blob()/pull path reads the whole object into RAM too So a multi-GB image needs multi-GB of RAM to add, and again to verify/pull -> OOM. Worse for images: omni stores the whole image as one inline blob PLUS (diskforge wiring) each guest file as its own blob. FIX (in omni-git, NOT bakery; bakery is being retired) Convert put_blob / put_file / get_blob / _blob_is_intact to STREAM in fixed-size chunks: a running hashlib.sha256 updated chunk-by-chunk, written to the .tmp file as it goes, and verified by streaming. Peak memory for blob I/O becomes ~one chunk regardless of file size. Fully backward-compatible: same sha256 addresses, same objects/ layout, same dedup+self-heal semantics. CONFIG (the RAM cap) Expose OMNI_CHUNK_BYTES (default ~8 MiB) as the chunk size = the working-set RAM ceiling for large-file ingest/pull. Unlike an external OS cap (which would just OOM-kill the current whole-file code), this lets big files SUCCEED within a set budget. OS/container limits (docker --memory, systemd MemoryMax, ulimit) remain a valid machine-level backstop. ACCEPTANCE / VERIFICATION * omni-git selftest stays 254/254 (no regression; addresses unchanged). * New pin: a large fixture ingests + pulls + verifies with PEAK RSS bounded ~ OMNI_CHUNK_BYTES (not file size); negative control = the old whole-file path blows the bound. * A large-file add succeeds under a tight OS memory limit that the current code OOMs on. SCOPE / CAVEAT Streaming bounds the FILE-BYTES memory (the huge-single-file case — the target here). It does NOT bound memory for an artifact with millions of tiny entries: manifest/index memory scales with ENTRY COUNT, a separate dimension (far smaller than file bytes). Note it; do not conflate. FILES: omni/store.py (put_blob, put_file, get_blob, _blob_is_intact; add streaming helper), possibly omni/cli.py for OMNI_CHUNK_BYTES surfacing/docs; README config table. Homes: GitHub dudezilla/omni-git + GitLab omni-git-entry-claude (kept consistent). Related: #493 (diskforge wiring), #495 (diskforge).

patch

none

approval

unapproved

runs

no runs recorded

mandate (clauses)

not yet recorded — lands with the ticket-class build

manifestations

not yet recorded — lands with the ticket-class build

guarantees

not yet recorded — lands with the ticket-class build

journey (blunders & successes)

not yet recorded — lands with the ticket-class build

artifacts (forge)

not yet recorded — lands with the ticket-class build

source (forge tree)

browse congruency source (forge-parked tree)

children

not yet recorded — lands with the ticket-class build