The Cognitive Processor
Cooperating HTTP services that implement sense → plan → act → score in software: a durable task tree, a sandboxed environment, an LLM-driven reasoning loop, a prediction market that scores outcomes in KL bits, and a recommendation engine that learns which tools, processes, and files belong in each step’s context — externalized sparse routing at organisation scale.
Boot a node
Stage 0 install — not a fractal layer. Use this once per host before §03 Quick start.
aion-mini is a lightweight tool-using LLM agent (single Python process)
that orchestrates the full install: build pot-cli,
join the governed PoT chain, start docker compose,
import baseline catalogs, start agent and predictor pools, and hand off a
core task (L3) plus core prediction (L4) to the Loop and chain.
| Stage | Component | Outcome |
|---|---|---|
| 0 | aion-mini | Bootloader agent runs boot_aion_core.md |
| 1 | pot.git | Rust toolchain (pot, git-pot) on PATH |
| 2 | PoT chain | Governed monorepo snapshot at ${AION_INSTALL_ROOT}/.pot-chain |
| 3 | aion-core | Compose stack healthy; bridge connected |
| 4 | Activation | Pools running; Loop claims the core task |
# optional goal — seeded as core task + core prediction at handoff:
aion-mini -p --append-system-prompt llm-programs/boot_aion_core.md \
"Your project goal here"
Install layout, activation checklist, and operator manual:
aion-core/docs/getting_started.md
·
bootloader spec:
aion-mini/llm-programs/boot_aion_core.md
·
aion-mini on GitLab
·
core prediction handoff:
blockchain guide §00
What it is
The formal model (Definition 2.1 of The Cognitive Processor) decomposes the agent stack into four disjoint subsystems: \(\Sigma_{\mathrm{int}} = \mathcal{P} \sqcup \mathcal{M} \sqcup \mathcal{L} \sqcup \mathcal{Q}\). aion-core maps these to real HTTP services:
| Symbol | Service | Port | Role |
|---|---|---|---|
| \(\mathcal{P}\) | Processor | 9205 | Durable task tree + synchronization primitives (SQLite) |
| \(\mathcal{M}\) | Machine | 9202 | Sandboxed filesystem + subprocess execution |
| \(\mathcal{L}\) | Loop | 9300 | LLM policy — merges state, calls model, dispatches tool calls |
| \(\mathcal{Q}\) | Prediction Market | 9400 | Per-task SUCCESS/FAILED markets; KL-scores forecasters |
Services communicate exclusively over REST/JSON. No shared Python imports cross boundaries at runtime. Each service is independently replaceable, scalable, and testable. An operator console (Svelte) at port 8090 provides a unified cross-service view.
Theory: persistence as objective
The natural objective of an agent stack is not "tasks completed" but persistence — keeping \(\mathcal{R}^{(\Sigma)} \ge 1\) over a time scale much longer than one Landauer cycle. Every term is measurable from internal logs:
Task selection under a finite compute budget is therefore a search problem: which READY task should be admitted to the RUNNING slot to maximise \(\mathbb{E}[\Delta\mathcal{R}]\)? The Cognitive Processor paper (Theorem 5.1) shows that MCTS over the task tree, with the LLM as proposer and per-task market beliefs as leaf evaluator, converges to the persistence-optimal selection at rate \(O(1/\!\sqrt{n})\) in the number of simulations \(n\):
- —Task tree in Processor \(=\) the MCTS search tree
- —LLM at temperature > 0 \(=\) stochastic expansion policy
- —Prediction market \(\hat{P}(\textsc{success})\) \(=\) calibrated leaf evaluator
- —Persistence ratio \(=\) the scalar to back-propagate
Theorem 5.2 proves that greedy (zero search — one LLM rollout) has a strictly shorter dissolution-time bound whenever the delusion divergence \(\mathcal{D}_{KL}^{(\Sigma)}\) exceeds a critical threshold. The processor's per-task market is the practical mechanism for detecting that threshold.
- → The Cognitive Processor — formal definition, MCTS theorems, aion-core instantiation
- → Information-Persisting Systems — the FPE that supplies the \(\mathcal{R}\) objective
docker compose up --build
# Traefik operator console → http://localhost:8090/
| Operator console | http://localhost:8090/ |
| Loop UI | http://localhost:8090/loop/ |
| Processor UI | http://localhost:8090/processor/ |
| Markets UI | http://localhost:8090/prediction_market/ |
| Chain bridge | http://localhost:9500/chain_status |
aion-core/.env)AION_LLM_API_KEY=sk-...
AION_LLM_MODEL_NAME=gpt-4o
# or Ollama: http://host.docker.internal:11434/v1
Services guide
One step: (1) fan out GET /state to all services and merge into one JSON; (2) build prompt from state + task message history; (3) call the LLM with OpenAPI-derived tools; (4) HTTP-dispatch tool calls; (5) append DeepDiff feedback; repeat. Two loop modes: messages (free-form tool calls) and state (durable agent scratchpad via loop_submit_step).
This is also the first compression layer: state mode, notes, and process hints reduce runtime context before any weight update is attempted. Multi-agent: AgentManager spawns typed AgentInstance workers, each with its own handler_id claiming READY tasks autonomously.
SQLite task tree with statuses PREPARING → READY → RUNNING → BLOCKED / COMPLETED / FAILED. Durable synchronization primitives: semaphore, mutex lock, event, timestamp lock, queue. Objects — named mutable context blobs that spawn child tasks on call. gather blocks a parent until listed children finish.
Exposes filesystem operations (read_file, write_file, glob, grep) and subprocess execution (exec) under a configurable DATA_ROOT. Path traversal is rejected. Per-task working directory is carried by the X-Task-State header from the processor.
Independent observer: processor webhooks on task message events trigger predictor bets. Market types: Bayesian (pseudo-count) and LMSR. Scoring: sequential KL reduction toward the final market distribution — same scoring identity as the blockchain. Predictors: random baseline, logistic regression (trained on historical tasks), LLM-based. Never mutates processor state.
Cross-cutting ranker: given task context, return top-k items per surface (tools, processes, files, docs, norms, markets, handlers). Embeddings in Postgres + Chroma; hybrid content + collaborative scores fused with legacy heuristics. Feeds the loop’s step appendix so the LLM sees a shortlist, not the full catalog. Surface weights auto-tune from \(\mathcal{D}_{KL}\) when engine-influenced paths correlate with better-calibrated outcomes.
Processes — soft workflow catalog; ranked suggestions on tasks, no mandatory BPMN.
Console — Svelte operator UI: tasks, agents, KPIs, onboarding wizard, company setup blueprint.
Blockchain bridge — maps task-level \(\mathcal{R}\) to PoT vote probabilities; talks to local chain via pot-mcp.
Context routing
MoE and attention sparsify computation inside the transformer. aion-core sparsifies what enters the prompt outside it.
A mature agent deployment exposes hundreds of tools, processes, and paths. Dumping all of them into every LLM step wastes tokens and invites wrong picks. The recommendation engine implements learned sparse attention over catalogs: embed the current task, score candidate items by vector similarity (and interaction history), return only top-k rows within declared token budgets.
| Inside FractalMoE (L1) | In aion-core (L3) |
|---|---|
| Token embedding | Embeddings service + Chroma item vectors |
| Attention / top-k keys | Recommendation engine rank per surface |
| MoE router → experts | Efficiency ladder: rule → script → distilled → LLM |
| Cross-entropy / surprise | State diff + task terminal status |
| \(\mathcal{D}_{KL}\) delusion tax | Prediction market KL scoring → surface weight updates |
The prediction market is the loss signal for routing. It does not choose tools or block tasks; it measures whether the system’s beliefs matched outcomes. When engine-ranked shortlists correlate with lower downstream \(\mathcal{D}_{KL}\), fusion weights rise; when they mislead, weights drop. Recommendations stay advisory — the processor and LLM remain authoritative.
North star: if the full inventory has \(N\) items and the engine returns \(k \ll N\), each step saves roughly \((N - k) \times\) average item tokens. Better routing is measured in KL bits, not click rate.
surface:tool— OpenAPI toolssurface:process— playbookssurface:file— machine pathssurface:doc— doc_api filessurface:norm— adopted normssurface:market— predictor queue
Use it in your own project
Loop speaks the OpenAI chat completions API. Point it at any compatible endpoint — OpenAI, Anthropic via proxy, local Ollama, or your own FractalLM instance:
AION_LLM_MODEL_NAME=llama3.2
AION_LLM_API_KEY=ollama
Each agent type has a model, system prompt, tool filter, loop mode, and parallel instance count. Define them in the operator console or via the Loop API. Agents claim READY tasks autonomously; task complexity gating controls which agent handles which task level.
Run FractalLM as the loop's LLM, enable the blockchain bridge, and task-level \(\mathcal{R}\) flows from token-level persistence → task metadata → PoT vote probabilities → on-chain trust. The full walkthrough is in round-trip.md:
docker compose up -d blockchain_bridge
curl http://localhost:9500/chain_status
Create versioned process templates in the Processes service. They attach as ranked suggestions to matching tasks — the LLM sees them as optional hints, not hard constraints. The Processes analytics track adoption rates and failure phases so you can improve playbooks based on agent behaviour, not guesswork.
Information stored
This is the main information-formation layer. The organisation interacts with reality through tasks, tools, outcomes, and reviews; those interactions are then recorded as structured traces. In other words, aion-core is where raw contact with the world becomes reusable information before it is later compressed into model weights.
| What | What it approximates | Storage medium |
|---|---|---|
| Task tree + message history | The primary trace of interaction with reality: what work was attempted, what was observed, and what reasoning led to each decision | SQLite — processor.db (WAL mode) |
| Agent type configs + trust scores | Which agent types exist, their capability profile, and how reliable each has proven to be | SQLite (loop DB) — AgentTrust.score, system prompts, model assignment |
| Norms + adoption markets | The organisation's current laws and the collective belief that each law improves \(\mathcal{R}\) | SQLite (norms DB) — norm records + norm:{id} market state |
| Prediction market state | The system's current approximation of task outcomes, continuously checked against what actually happens | SQLite (market DB) — bets, Bayesian/LMSR state, forecaster scores |
| Processes + playbooks | Versioned templates for repeatable work — the organisation's distilled procedural knowledge | SQLite (processes DB) + YAML seeds on disk |
| Learnings | Compact lessons extracted from real task failures, reviews, and conflicts — already halfway to training data | Markdown — learnings/<agent_type_id>.md via doc_api |
| Machine sandbox files | Artifacts produced by acting in the world — code, reports, datasets, evaluations, and other evidence of contact with reality | Filesystem under DATA_ROOT (per-task cwd) |
| Embeddings + vector index | Semantic similarity over processes and docs — enabling retrieval without full-text search | Postgres + Chroma (recommendation engine) |
| Persistence metadata on messages | Token-level \(\mathcal{R}\) from aion-llm — the model-side signal fused with task outcomes to form richer supervision | Task message JSON field (message.persistence) in processor DB |
Delusion (\(\mathcal{D}_{KL}\)) when stale. The organisation's approximation degrades when tasks are poorly instrumented, when outcomes are not resolved honestly, when processes drift away from actual work, or when norms are adopted without contact with outcomes. The prediction markets are the primary mechanism for detecting this gap: a market that stays near 0.5 signals a domain where the system has not yet formed reliable information from its reality contact.
What flows up and down. Upward, the blockchain bridge maps task-level \(\mathcal{R}_T\) and outcome evidence to PoT vote probabilities, turning local experience into society-level trust updates. The operator’s core task (what to do) pairs with a core prediction on the chain (what must be true) — see blockchain guide §00. Downward, chain-level norms, trust changes, core predictions, and adopted patches flow back into core as foreign norms, policy constraints, and training material. From there, curated traces can be exported again into aion-llm as stable `train.jsonl`, `val.jsonl`, and `replay.jsonl` corpora for SFT-first checkpoint distillation, with LoRA still planned.