Every ticket as the root of its provenance graph — goal, patch, approval, runs and (as the ticket-class build lands) mandate, manifestations, guarantees, journey, artifacts and children. Rendered through the <<<TicketGraphList>>> tag.
goal
Rewrite ONE unique forge module as entirely new, behavior-preserving, installable,
control-config'd Python, per the FORGE-wide refactor standard #200 (#std).
This ticket covers ONE module — the one identified below — and nothing else. It was filed
against the module's DEDUPLICATED identity, so if that module was copy-pasted across the
forge, every copy is covered by this single ticket.
MODULE KEY : forge-rw:3b270a86c0a63f8f
(the ticketer's idempotency marker — it lives in this ticket's TITLE. Do
not edit the title, or the module will be re-filed as a duplicate.)
IDENTITY : AST-NORMALIZED — codebean's `_ast_norm` (#183), i.e.
ast.dump(ast.parse(ast.unparse(ast.parse(src)))).
Copies of this module that differ only in comments, blank lines,
quoting or spacing carry the SAME module-sha and are covered by THIS
ticket. A differing docstring is real content, so a module whose
docstring differs is a DIFFERENT module with its own ticket.
SIZE : 15703 bytes
EXTENSION : .py
CANONICAL SOURCE (the location this ticket is named for):
repo : agent_tools_suite
path : gate.py
ref : HEAD (HEAD)
blob : 0de4e67af9d14d6fbacdbd002c567f8f00586c65 (git blob sha1)
REFERENCE ONLY — you do NOT need this command. This module's bytes are embedded
COMPLETE in THE ORIGINAL SOURCE block below, and that is what the rewrite is based
on. For a reader who does have forge access:
git --git-dir=/Users/stevenpeterson/code/installed/jazz-project/config/forge/remotes/agent_tools_suite.git cat-file blob 0de4e67af9d14d6fbacdbd002c567f8f00586c65
THE SAME MODULE ALSO LIVES AT THESE FORGE TIPS: none — this module is unique to
agent_tools_suite among the repos scanned so far.
HISTORICAL OCCURRENCES of this exact blob — every OTHER DISTINCT {repo, path}
the omni-git index (`occ`, joined on the git blob sha1) records this
byte-identical file at, with the index's own `ts` for the first row of each.
Any place this run enumerated LIVE is listed under the forge tips above, not
here. Context for the rewrite, not extra work:
testmonkeyalpha-group/jazz-forge/agent_tools_suite:gate.py [occ.ts gitlab]
THE ORIGINAL SOURCE, EMBEDDED — YOU NEED NO FORGE ACCESS TO DO THIS TICKET.
The module's own bytes are reproduced below, inside this ticket. THEY are the
authoritative copy and the behavior-preserving basis: rewrite from them. The
`git --git-dir=... cat-file blob` command under CANONICAL SOURCE above is kept only as a
REFERENCE for a reader who happens to have forge access — running it is NOT part of this
ticket. (Why the bytes are here at all: the pair_loop worker that executes this ticket is
sandboxed to the ticket's own congruency worktree and cannot read the forge's bare repos.
Ticket #346 is the fix that put them on the ticket.)
The block BEGINS at the line
===ORIGINAL SOURCE (verbatim, the behavior-preserving basis)===
and ENDS at the first line
===END SOURCE===
Everything strictly between those two lines is the module — 15703 byte(s), byte for byte, with no re-indentation, no re-wrapping and nothing
elided.
===ORIGINAL SOURCE (verbatim, the behavior-preserving basis)===
#!/usr/bin/env python3
"""gate — the top MCP server. Owns the coupler.
Everything Claude does through the aggregator flows Claude -> gate -> coupler ->
downstream. The gate exists to supply the two things the model cannot do itself:
1. TIME — it stamps every routed call with the real system clock. (The model's own
sense of time is a hallucination, so it must never supply a timestamp.)
2. MEMORY / INTROSPECTION — it records each {timestamp, tool, intent} as a memory in
a JSON log in ~/.MCP, and exposes gate__query so the model can look back at what
it used, when, and why.
To route any aggregated tool the call must carry an `intent`; the gate logs it and only
then opens the coupler. Meta tools: gate__now, gate__query, gate__status.
"""
import json
import os
import subprocess
import sys
import urllib.request
from datetime import datetime, timezone
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
import mcplib # noqa: E402
_reg = mcplib.load_registry(HERE).get("gate", {})
LOG = os.path.normpath(os.path.join(HERE, _reg.get("log", "gate-memories.json")))
_downstream_spec = _reg.get("downstream", {"command": "python3", "args": ["coupler.py"]})
# --- persistence: memories go into the ONE persistent install db (~/.MCP/project.db via store.py). --- #
# The JSON log (LOG) is kept ONLY as an emergency fallback so a db outage never breaks routing or drops
# a memory. `store` is imported best-effort — if the tracker tree is absent (headless), the gate still runs.
_TRACKER_DIR = os.path.normpath(os.path.join(
HERE, os.environ.get("TRACKER_DIR", "../packages/tracker")))
try:
sys.path.insert(0, _TRACKER_DIR)
import store as _store # noqa: E402 — the project DB layer (memories table lives here)
except Exception: # noqa: BLE001 — never let a missing tracker tree stop the gate
_store = None
# The real Claude Code session id, so memories persist AS sessions. Fallback keeps a stable key headless.
SESSION_ID = os.environ.get("CLAUDE_CODE_SESSION_ID") or ("pid-%d" % os.getpid())
coupler = mcplib.Downstream("coupler", _downstream_spec, HERE)
_started = False
def now_iso():
return datetime.now(timezone.utc).isoformat()
def _read_log():
if os.path.exists(LOG):
try:
with open(LOG) as f:
return json.load(f)
except Exception:
return {}
return {}
def _append_json(entry):
"""Emergency fallback only: append the memory to the JSON log (atomic write)."""
data = _read_log()
key = entry["timestamp"]
while key in data: # never clobber a same-instant memory
key += "-x"
data[key] = entry
tmp = LOG + ".tmp"
with open(tmp, "w") as f:
json.dump(data, f, indent=2)
os.replace(tmp, LOG)
def _record_db(entry):
"""Write the memory to the persistent install db (project.db `memories`). Returns True on success."""
if not _store:
return False
try:
_store.record_memory(tool=entry.get("tool"), intent=entry.get("intent"),
session=entry.get("session"), note=entry.get("note"),
ts=entry.get("timestamp"))
return True
except Exception: # noqa: BLE001 — a db hiccup must never break routing; fall back to JSON
return False
def append_memory(entry):
entry.setdefault("session", SESSION_ID)
recorded = _record_db(entry) # primary: the persistent tracker db
if not recorded:
_append_json(entry) # fallback: never lose a memory if the db is unreachable
_cong_memory(entry) # forward the provenance to the running congruency (best-effort)
# --- congruency provenance forward -------------------------------------------------------------- #
# Follow the rabbit trail from prompt to action in the RUNNING congruency: register this agent
# (id + prompt + tools) once, then mirror every {session,tool,intent} into congruency `memories`,
# so `?route=agent.trail&agent_id=<session>` replays exactly what this session did and why. All of
# it is best-effort — a congruency outage (or a missing key) never blocks or breaks routing.
_CONG = _reg.get("congruency") or {}
_CONG_URL = (os.environ.get("CONGRUENCY_URL") or _CONG.get("url") or "").rstrip("/")
_CONG_KEY = None
_cong_registered = False
def _cong_apikey():
global _CONG_KEY
if _CONG_KEY is not None:
return _CONG_KEY or None
key = os.environ.get("CONGRUENCY_API_KEY", "")
if not key:
secret = _CONG.get("key_secret", "congruency-api-key")
for wallet in ("wallet", os.path.expanduser("~/bin/wallet")):
try:
r = subprocess.run([wallet, "decrypt", secret], capture_output=True, text=True, timeout=8)
if r.returncode == 0 and r.stdout.strip():
key = r.stdout.strip()
break
except Exception: # noqa: BLE001 — wallet absent/locked: forward stays disabled
continue
_CONG_KEY = key
return key or None
# Hardened forward: a gated call must not silently vanish from congruency. Every forward is either
# delivered or PERSISTED to a durable replay queue and re-sent on a later call — so a congruency
# outage (or a slow 2s timeout) queues instead of dropping. The api key is sent when we have one but
# is NOT required (congruency's CONGRUENCY_AUTH_OVERRIDE accepts keyless writes), so a locked/absent
# wallet no longer disables the forward. All best-effort — never raises, never breaks routing.
_CONG_PENDING = os.path.join(HERE, "cong-pending.jsonl") # gitignored replay queue
_CONG_TIMEOUT = 2
_CONG_FLUSH_MAX = 25 # bound per-call latency draining a backlog
_CONG_QUEUE_MAX_BYTES = 1000 * 1024 * 1024 # cap the replay queue UNDER 1000 MB
def _cong_send(query, obj):
"""One POST to congruency. Sends X-Api-Key iff a key resolves (not required). Raises on failure."""
headers = {"Content-Type": "application/json"}
key = _cong_apikey()
if key:
headers["X-Api-Key"] = key
req = urllib.request.Request(_CONG_URL + query, data=json.dumps(obj).encode(), method="POST", headers=headers)
urllib.request.urlopen(req, timeout=_CONG_TIMEOUT).read()
return True
def _cong_enqueue(query, obj):
try:
with open(_CONG_PENDING, "a") as fh:
fh.write(json.dumps({"q": query, "o": obj}) + "\n")
_cong_trim()
except OSError:
pass
def _cong_trim():
"""Bound the replay queue under the cap. A congruency outage that never ends must not grow the
queue without limit — when it exceeds the cap, DROP the oldest entries (telemetry favors recent),
trimming to ~90% for hysteresis so we don't rewrite on every enqueue. Never raises."""
try:
if os.path.getsize(_CONG_PENDING) <= _CONG_QUEUE_MAX_BYTES:
return
lines = [l for l in open(_CONG_PENDING).read().splitlines() if l.strip()]
except OSError:
return
target = int(_CONG_QUEUE_MAX_BYTES * 0.9)
total = sum(len(l) + 1 for l in lines)
i, dropped = 0, 0
while i < len(lines) and total > target: # drop from the FRONT (oldest) until under target
total -= len(lines[i]) + 1
i += 1
dropped += 1
keep = lines[i:]
try:
with open(_CONG_PENDING, "w") as fh:
fh.write("\n".join(keep) + ("\n" if keep else ""))
if dropped:
sys.stderr.write("[gate] cong replay queue over %d MB — dropped %d oldest\n"
% (_CONG_QUEUE_MAX_BYTES // (1024 * 1024), dropped))
except OSError:
pass
def _cong_flush():
"""Replay the queued forwards oldest-first; stop at the first failure (congruency still down) and
keep the remainder for next time. Bounded per call. Never raises."""
if not _CONG_URL or not os.path.exists(_CONG_PENDING):
return
try:
lines = [l for l in open(_CONG_PENDING).read().splitlines() if l.strip()]
except OSError:
return
remaining, sent = [], 0
for i, line in enumerate(lines):
if sent >= _CONG_FLUSH_MAX:
remaining.extend(lines[i:]); break
try:
rec = json.loads(line)
except Exception: # noqa: BLE001 — drop a corrupt line
continue
try:
_cong_send(rec["q"], rec["o"]); sent += 1
except Exception: # noqa: BLE001 — still unreachable: keep this + the rest, in order
remaining.extend(lines[i:]); break
try:
if remaining:
with open(_CONG_PENDING, "w") as fh:
fh.write("\n".join(remaining) + "\n")
else:
os.remove(_CONG_PENDING)
except OSError:
pass
def _cong_post(query, obj):
"""Durable best-effort forward: drain any backlog (chronological), then send this one; on failure
persist it for replay so it is never lost. Never raises."""
if not _CONG_URL:
return False
_cong_flush()
try:
_cong_send(query, obj)
return True
except Exception: # noqa: BLE001 — congruency down/timeout: queue, don't drop
_cong_enqueue(query, obj)
return False
def _cong_memory(entry):
if not entry.get("tool"):
return
_cong_post("/?api=memories", {"ts": entry.get("timestamp"), "session": entry.get("session", SESSION_ID),
"tool": entry.get("tool"), "intent": entry.get("intent"),
"note": entry.get("note")})
def _cong_register():
"""Register this session as an agent (id + prompt + granted tools) in congruency, once."""
global _cong_registered
if _cong_registered or not _CONG_URL:
return
_cong_registered = True # attempt once regardless of outcome (best-effort)
tools = ",".join(sorted(t.get("name", "") for t in coupler.tools)) if not coupler.dead else ""
_cong_post("/?route=agent.register",
{"agent_id": SESSION_ID, "parent": os.environ.get("CLAUDE_PARENT_SESSION_ID", ""),
"prompt": os.environ.get("CLAUDE_CODE_PROMPT", ""), "tools": tools})
def connect():
global _started
if _started:
return
coupler.start()
_started = True
_cong_register() # register the agent + its toolset with congruency (best-effort)
GATE_META = [
{"name": "gate__now",
"description": "[gate] Authoritative current time (ISO-8601 UTC). Use this; never invent time.",
"inputSchema": {"type": "object", "properties": {}, "additionalProperties": False}},
{"name": "gate__query",
"description": "[gate] Query recorded tool-use memories (timestamp, session, tool, intent) from the persistent db. Optional 'contains' substring filter, 'session' (an id, or 'current' for this session), and 'limit'.",
"inputSchema": {"type": "object",
"properties": {"contains": {"type": "string"},
"session": {"type": "string"},
"limit": {"type": "integer"}},
"additionalProperties": False}},
{"name": "gate__status",
"description": "[gate] Gate health + the coupler it owns.",
"inputSchema": {"type": "object", "properties": {}, "additionalProperties": False}},
]
def _inject_intent(schema):
s = dict(schema or {"type": "object"})
props = dict(s.get("properties", {}))
props["intent"] = {"type": "string",
"description": "Why you are calling this tool right now (required — the gate records it)."}
s["properties"] = props
req = list(s.get("required", []))
if "intent" not in req:
req.append("intent")
s["required"] = req
return s
def list_tools():
connect()
if not coupler.dead:
coupler.refresh_tools() # reflect the coupler's CURRENT downstream set
tools = list(GATE_META)
if not coupler.dead:
for t in coupler.tools:
nt = dict(t)
nt["inputSchema"] = _inject_intent(t.get("inputSchema"))
nt["description"] = ("%s [gate: intent required]" % t.get("description", "")).strip()
tools.append(nt)
return tools
def _query(args):
args = args or {}
contains = args.get("contains")
session = args.get("session")
if session == "current":
session = SESSION_ID
limit = int(args.get("limit") or 50)
# primary: the persistent db (session-aware). Fall back to the JSON log if the db is unreachable.
if _store:
try:
rows = _store.memories(session=session, contains=contains, limit=limit)
rows = list(reversed(rows)) # store returns newest-first; show oldest→newest like the log
if not rows:
return mcplib.text_result("(no memories recorded%s)" %
(" for session %s" % session if session else ""))
lines = ["- %s [%s] %s — %s" % (r.get("ts"), (r.get("session") or "-"),
r.get("tool"), r.get("intent")) for r in rows]
return mcplib.text_result("## gate memories — %d shown (db: %s)\n\n%s" %
(len(rows), getattr(_store, "DB", "project.db"), "\n".join(lines)))
except Exception: # noqa: BLE001 — degrade to the JSON log rather than error the query
pass
data = _read_log()
items = sorted(data.values(), key=lambda e: e.get("timestamp", ""))
if session:
items = [e for e in items if e.get("session") == session]
if contains:
c = contains.lower()
items = [e for e in items if c in json.dumps(e).lower()]
items = items[-limit:]
if not items:
return mcplib.text_result("(no memories recorded yet)")
lines = ["- %s [%s] %s — %s" % (e.get("timestamp"), (e.get("session") or "-"),
e.get("tool"), e.get("intent")) for e in items]
return mcplib.text_result("## gate memories — %d shown (log: %s)\n\n%s" % (len(items), LOG, "\n".join(lines)))
def call_tool(name, args):
if name == "gate__now":
return mcplib.text_result(now_iso())
if name == "gate__query":
return _query(args or {})
if name == "gate__status":
connect()
state = "✗ down (%s)" % coupler.error if coupler.dead else "✓ up, %d tools" % len(coupler.tools)
return mcplib.text_result("## gate — owns the coupler\nlog: %s\ncoupler: %s" % (LOG, state))
# any other name = a routed aggregated tool → gate it on intent, stamp time, record
connect()
args = dict(args or {})
intent = args.pop("intent", None)
ts = now_iso()
if not intent:
append_memory({"timestamp": ts, "tool": name, "intent": None, "note": "REFUSED: no intent"})
return mcplib.text_result(
"🚪 gate: intent required. Re-call '%s' with an 'intent' saying why. (logged %s)" % (name, ts), True)
append_memory({"timestamp": ts, "tool": name, "intent": intent})
if coupler.dead:
return mcplib.text_result("gate: coupler is down (%s)" % coupler.error, True)
try:
return coupler.call_tool(name, args)
except Exception as e:
return mcplib.text_result("gate: coupler failed on %s: %s" % (name, e), True)
if __name__ == "__main__": # importable (tests drive the forward helpers) without serving
mcplib.Server("gate", "1.0.0", list_tools, call_tool, watch_tool_changes=True).serve()
===END SOURCE===
THE CALLABLE SURFACE — every callable in the original, lowered with codebean's
`lower_node` (#183). The rewrite must present THIS surface (same names, same
parameters, same defaults) so callers of the original keep working:
now_iso()
_read_log()
_append_json(entry)
_record_db(entry)
append_memory(entry)
_cong_apikey()
_cong_send(query, obj)
_cong_enqueue(query, obj)
_cong_trim()
_cong_flush()
_cong_post(query, obj)
_cong_memory(entry)
_cong_register()
connect()
_inject_intent(schema)
list_tools()
_query(args)
call_tool(name, args)
CAPTURED BEHAVIORAL EXEMPLARS: none. no function in this module is minable — criteria_miner mines only
self-contained pure functions (it refuses anything that mentions a
world-touching or non-deterministic name, plus methods and no-arg
functions), and nothing here qualified
This does NOT weaken the ticket's bar — it MOVES the work: you must write the
equivalence exemplars by hand from the original's behavior before rewriting, and
they are what settles this ticket.
THE STANDARD — #200's four requirements, which this ticket exists to enforce. All four are
the standard's own words; none is optional:
1. BEHAVIOR-PRESERVING, VERIFIED NOT ASSERTED — the deliverable is ENTIRELY NEW source
code (rewritten, not copied). It must reproduce the ORIGINAL's behavior, and the proof
is the exemplars above (captured via codebean lower_* #183 + criteria_miner
IO-exemplars, exactly as #200 specifies).
2. A PYTHON MODULE — regardless of the source language. This module's source is `.py`;
the rewrite is Python either way.
3. INSTALLABLE — a proper package: pyproject.toml, `pip install -e .` works, console
entry points where applicable. WRITE ONLY YOUR OWN MODULE FILE at
forge2/<repo>/<module>.py. Do NOT hand-author pyproject.toml or __init__.py: the
harness DERIVES them ADDITIVELY from the modules present (congruency/tools/
forge_package.py, #410 — the analogue of the tag-registry fix #281), so your unit never
conflicts with, or clobbers, a sibling module of the same forge2/<repo>/ package. A
package-level pyproject includes every .py in the directory, so your module installs
and imports as `<repo>.<module>` the moment it lands.
4. CONTROL-CONFIG'D — NO hardcoded paths or params. A JSON/registry config object drives
it (the install.json / registry.json idiom of this repo), and per note-for-claude a
tool that cannot see its config THROWS rather than guessing a default. Add ONLY your
own `forge2.<repo>.<module>` section to the package's registry.json (create the file if
it is absent); NEVER read, edit or drop a sibling module's section — the sections are
disjoint keys and stay additive that way.
ACCEPTANCE (#200's, per rewrite ticket) — this ticket settles when ALL of:
a. the new source reproduces every captured exemplar above, plus the edge cases you add
for what sampling cannot reach (paste the runs as evidence);
b. it presents the callable surface above, so existing callers keep working;
c. `pip install -e .` installs it clean from its own pyproject.toml;
d. it runs off a config object with ZERO hardcoded settings, and throws without one.
Kind is `build` (a pair: coder + adversarial tester). This ticket hangs off #200 by a
`spawned` edge, so a stop ticket aimed at #200 brakes every rewrite in this programme.
Filed mechanically by checkouts/current/congruency/tools/forge_refactor_ticketer.py under ticket #201, from the forge bare repos at
/Users/stevenpeterson/code/installed/jazz-project/config/forge/remotes. Its original source is embedded above (#346), so this ticket is complete on its own.