MeshWeaver is a distributed platform for building data-driven applications with AI capabilities. A handful of principles hold the whole system together:

Platform overview

Core concepts

Topic map

Each theme starts with its introductory page, followed by related architecture topics.

Reactive core

• Start here: Asynchronous Calls — the no-await rulebook
• Actor Model
• Turn-Loop Arrival Order — a hub has TWO queues and a delivery moves between them at turn time, so a turn can be in neither when the last gate opens; the permutation names which message straddled the window, never which defect is live
• Controlled I/O Pooling
• Subscription Ownership — a pending timer is a GC root
• Silent Completion — an empty completion is invisible to every timeout
• AsyncLocal Across Scheduler Hops
• Initialization Gates
• What the DataContext Init Time-Box Bounds — the 120 s box is nested three deep (data source → stream → type-source leg), a per-node hub's initialization is in practice ONE unbounded storage read, and the timeout now names the leg instead of guessing "a stuck NodeType compile", which cannot reach it
• The Initialization Budget Ladder — initialization is a nested wait and every level took the same 120 s constant; equal is not an ordering, so when the enclosing bound won the level that knew which wait starved was torn down as a recognised shutdown and recorded nothing. A hub born inside another hub's initialization now takes a strictly contracting rung — and HOSTED is not ENCLOSED, so a per-node hub keeps the full 120 s for its whole initialization while its own time-box, one rung in, reads 115 s
• Retiring an Activation — a transient init fault retires instead of latching; the gate is failed BEFORE the dispose, because two drains cannot be ordered by a comment
• Stale State Until a Recycle — an activation serves what it bound and never re-reads it, so merged, sealed, rolled and restarted do not make a fix live at an address that is already up; what a DisposeRequest changes and what it provably does not
• Aggregating Providers
• Hub Disposal Model
• Transient Node Probes — a probe hub's own address is not a node
• Executive Assistant Credential Reads — a turn that waits for a mesh read holds the queue that read's reply must travel through, and the timeout was rendered as "you never connected"
• Bounds Must Be Ordered
• Message-Based Communication
• A Request a Hub Sends to Itself — node CRUD is issued on the hub that executes it, so routing and the reply leg cannot lose it; what that leaves, and why an empty queue at the timeout proves nothing when the handler answers from a detached observable
• Router Traffic Detection — the detector has two sites; the receiver names the two addresses, the origin names the call site, and a ratchet per tree keeps the seams adopted
• No Static State
• The Routing Entry Point — every routed message pays an Orleans grain PLACEMENT; the two measured ways it fails and drops live traffic, and the case for a local routing service where a silo is co-hosted
• Open Vocabularies Are String Constants — a persisted or module-extended vocabulary is const string, never an enum; it stays open for anyone to extend with their own constants, and resolution is a durable chain of rule nodes rather than a switch
• Observable Hub Pipeline (migration design)
• Per-Hub TaskScheduler — Actor Isolation Across the Mesh
• Removing Hand-Woven Concurrency Gates
• Removing Observable-to-Task Bridges
• A Bounded Fan-Out Must Not Recurse — Rx's Merge(n) subscribes the next queued inner inside the previous one's OnCompleted, so synchronously-completing inners (a request from a hub at ShutDown, a .Catch(→ Return)) grow the stack per queued item until the process dies of an uncatchable StackOverflowException; MergeBounded(n) starts the next inner from a drain loop instead
• JSON Serialization

Reading & writing nodes

• Start here: CQRS — Queries vs. Content Access
• Three Registries a Read Can Miss — "the type is not registered" is three unrelated tables (the hub's type registry, the workspace's mapped collections, the reduce manager), so a guard that checks one while the read uses another passes having checked nothing; plus what a refusal and a recovery failure must NAME to be followed up
• An Answer Nobody Gave Is Not Cached — a synced chain replays its FIRST frame for the life of the process, and a provider that completes without an Initial is counted as an empty one, so a cold moment used to become a permanent false "absent"; the frame now names who never answered and an unanswered frame is delivered but not kept
• The Query Fan-In's Stall Terminal — a provider that neither emits, completes nor errors used to starve the all-providers Initial gate for ever, with no error and nothing to grep; the merge now faults naming it, at a DERIVED rung strictly inside the permission fold's own budget, and every consumer that decides access already fails closed on it
• A Standing Watch Re-Opens After a Transient Fault — a subscription held for the process's life has no next caller, so it must re-open itself after a typed transient fault, bounded; and "no answer in time" has two shapes, the in-process timeout and the router's flattened one
• MeshNode Stream Cache
• Update Queue Ownership — one published queue per path, retained until accepted work settles
• Request via Stream Update
• Expressing a Write — the four shapes a mutation can take (C# lambda, JSON Patch + text splice, full entity, other), which context may use which, and the lowering that keeps the fold owner-side
• Data Access Patterns
• Data-Plane Messages Are Stream Plumbing — GetDataRequest/DataChangeRequest and their kin are the synchronization streams' wire format, not an API: read a node with GetMeshNodeStream, write it with .Update, create it with IMeshService; the inventory of every remaining caller, the per-kind recipe, the shrink-only ratchet, and the target of all five types internal to the data layer
• Node Identity and Path Keying — (namespace, id) is the key and path is derived, so splitting a path positionally leaves the path identical while re-keying the node into a second row
• Workspace References
• Content Chunk Navigation
• Moving Nodes — a move relocates the node and everything that belongs to it, or it refuses
• Copy Completeness — a copy asserts a set equality it never established; the two readings that turn a short enumeration from a silence into a number, and why the failure stops instead of rolling back
• Moved-Node Redirects — keeping links alive after a move
• MainNode and Rebasing — a with { Namespace = … } copy un-lists a node with nothing logged
• Write Verdict Totality — a write whose base read ends empty answers nobody, and arms no deadline either
• Reading a Base-State Timeout — "no initial state arrived within 30s" has two causes with two different fixes; the census that says which, and how to read one off a running deployment
• The Phantom Base After Owner Disposal — the owner echoes a merge BEFORE it flushes it, so a "never applied" NACK's re-attempt can find its own unpersisted write in the mirror, diff to nothing and report success
• Conditional Writes Across Hubs — on a node you do not own the lambda runs on YOUR mirror and ships a diff, so a field it decided not to write is absent from the patch and a concurrent write to it survives
• Live Mirrors and the Change Feed — a write must not end its own streams
• Stream Liveness and the Hub Reference — a stream outlived the hub it held, and the contract said it could not
• The sync/ Hub Population — a sync/ hub is one field of a stream and a subscription makes two of them; the only reaper of a Started one is an idle sweep the traffic keeps re-arming
• A Hub That Pins Its Own Cache Entry — on Orleans every activated hub held the cache's live view of its own path for life, so the entry never released, its heartbeat kept the grain alive, and the loop closed; the one-shot own-node source that opens it
• The Evicted-Stream Retention — a change-feed eviction parks a remote stream and ReclaimIfUnheld refuses to dispose one that carries no lease entry, so every unleased call site retains one stream, and two sync/ hubs, per change event
• The Read Path Minted a Hub Per Read — a live in-process census decomposed a replica's sync/ hubs into their holders and pinned the growth on the read path: a constant configuration took GetDataRequest out of the stream cache, so every read left a permanent hub behind (six reads, six hubs, measured on the running portal)
• A Reference That Cannot Be a Key — the same defect through the other door: a record whose member is a collection is compared BY REFERENCE, so the reference can never hit the stream cache at all; every one of a replica's 311 sync/ hubs attributed to its minting stream, and the duplicates split into "cache bypassed" and "key unhittable" by comparing the reference objects on the heap. Carries the census script, because the last two were lost
• A Release Refused at the Source — the other direction: why the population never DRAINS. An UnsubscribeRequest is the only thing that ends an owner-side sync/ hub, and on the hub-teardown route the subscribing hub is in DisposeHostedHubs by construction when it is posted, so the teardown guard refused it at the source and the owner was never told; the IReleasesRemoteState marker and forwarding through captured parents to a surviving ancestor, preserving the fully qualified sender
• The Recursive-Delete Drain — the plan is a snapshot the removals may exceed, the completion check must include the ROOT, and the stage bound measures progress, not duration
• Deleting What Is Already Gone — an absent node already satisfies the delete's postcondition, so the delete succeeds and reports that it removed nothing; why checking existence first cannot close the race, and which absences are still failures
• Business Rules & Calculations
• Data Versioning Strategies
• Mesh Graph Architecture
• MeshNode Versioning
• Query Provider Parity
• A Failed Read Is Not An Absent Node — a store fault caught inside a query provider arrived downstream as a COMPLETE answer, so a read that failed reported a node that does not exist and a per-node hub was refused over a working node; the one provider-side verdict that closes it, why it folds into SilentProviders instead of adding a second decider, and the two sides a test of it must have
• Search Coverage and Refusal — the search tool answered an unanchored query with a clean 0 for nodes it returned when anchored; it now refuses a query that names no partition and declares no fan-out, and every envelope carries coverage.partitions — the denominator a zero is read against
• Query Result Scoring
• Reading a Write Verdict
• Satellite Entity Patterns
• Satellite Node Patterns
• Repairing a Stale MainNode — When the Broken Field Guards Itself
• Synced Mesh Node Queries
• Update Validators See Typed Content
• Content Is Validated Against Its Declared Shape On Write — content that cannot bind to its NodeType's declared content type was stored verbatim and then read as absent everywhere; the two shapes the write boundary refuses, the narrow rule that keeps legitimate writers landing, and the wire-boundary residue it does not close
• The /api/mesh REST Contract — a 200 carries the verb's JSON document and nothing else; a sentinel answer is a non-2xx JSON envelope naming the sentence and its kind

Storage & partitions

• Start here: Postgres Schema Architecture
• Partition Storage Routing
• Storage Bindings — where a partition keeps its data: zero bindings = the instance default; a binding at {partition}/_Storage (global: Admin/_Storage) picks or creates a container in one of the instance's pre-configured stores, validated on the node; the resolver every storage consumer asks; the Storage settings section
• Partition Teardown — deleting a partition ROOT drops its backing store; keyed on the node's SHAPE, never its NodeType
• Partition Storage Hubs
• Partitioned Persistence
• Storage Adapter Implementation
• Change-Feed Isolation — one throwing subscriber must never starve the others
• Change-Feed Gaps — a reconnected LISTEN session has lost every commit in between; the backend declares the gap and each cache re-reads its authoritative state (Plugins#3000)
• In-Memory Child Index Consistency — a listing taken while the in-memory store re-indexed came back short, a synced query cached it for good, and one NodeType's compile then failed on files that were there; the three rules that keep a reader from ever seeing a half-built index, and how it composes with the mid-install judgement race (#4280)
• A Container Registry in Memex — the fleet's own registry at cr.meshweaver.cloud (a separate distribution + docker_auth service), the bootstrap circularity that keeps the hosting instance's boot image off it, and the in-portal mirror that was built, never wired, and deleted (#4066)
• Static Repo Import
• The Prune Requires a Complete Listing — an import prunes on "absent from the source ⇒ deleted"; a truncated GitHub tree arrives as HTTP 200 and turns every unread file into a deletion
• Import Write Ordering — a NodeType lands before the instances that name it
• Import-Side Content Degradation — a typed file whose own parser is absent on the importing host lands as Markdown with its configuration discarded, and the read-side degradation instrument is blind to it by construction, because the content it wrote types perfectly (#4319)
• Vector Search
• Durable But Unreadable — a write that is acknowledged, versioned and invisible
• Instance-Key Resolution — the registry reads an instance key through live mirrors, never a per-request point read
• Cross-Schema Fan-Out Elimination — an unanchored query is a lock bomb; the census and the per-caller plan
• Addressed Notifications — plan 1 worked out: deliver a notification to its addressee so the bell reads two schemas, not 199
• Content Indexing Activation
• Document Parts — logs, transcripts and files as one Document node plus indexed DocumentPart nodes in their own partition table, written chunk by chunk
• Cross-Instance Mirror
• Data Synchronization and CRDT
• Setting Up Data Sync
• DatabaseBackups
• Declarative export and import
• Syncing a Space with GitHub
• What a Green Build Costs a Synced Space — one field decided whether a delivery was free or a full clone, and it is deliberately frozen while an import does not converge; the second, weaker pointer that makes a settled source free again, and why the skip needs the verdict to be FINAL and not merely recorded
• When a Publication Seal Stops Advancing — a Space converges on a green build only while the publication sealed for THIS instance's framework identity keeps reaching the built commit; when the instance's identity and the lane that publishes for it drift apart that condition stops being satisfiable, and until the hold was recorded on the node a held source was byte-identical to a settled one
• The Import Marker Records Convergence
• A Content Verdict Is Per Node — one unstorable byte in one file earned a partition-wide "these bytes cannot import" verdict and took every instance action offline for an evening; where the refusal is remembered now, and what an import must name when it loses a node
• A Parked Type Names the Import — a NodeType failing on CS0246 for a symbol whose file is plainly in git is an IMPORT state, not a code one; how the compile failure is joined to the import that lost the file, why the join is scoped to the types that actually reference it, and the three answers it keeps apart so an import is never accused of a deletion it did not make
• Instance Sync — bi-directional space replication between MeshWeaver instances
• Managing Partition Sync (Admin Guide)
• Static Node Providers

Security

• Start here: Access Control
• Granting Access
• AccessContext Propagation
• Query Identity — an unstamped read answers as Anonymous, which reads as absence
• Participant Ingress — SignalR and gRPC forward any delivery to any address with a client-written envelope; mesh infrastructure is [InfrastructureOnly], every ingress delivery is stamped, and the receiving hub refuses a stamped infrastructure message (SaveMeshNodeRequest, partition storage, compile triggers) before any handler runs
• Owner Injection
• Authorize as Caller, Execute as System — a control-plane operation checks the caller's access explicitly when the request is written and again before it runs, then executes as System; never depends on the caller's RLS reach mid-execution
• Sole-Maintainer Approval — every approval is four-eyes except the installation's declared maintainer approving their own request: one rule for instance actions, operation requests and governed-activity signatures, stamped and shown, plus the approvals inbox that approves in bulk through each item's own path
• Permission API
• Settings by Owner — the Instance, Person and Node Apps — every settings tab lives in the app of the thing it changes: /Admin (the instance, titled with its name, platform admins only), /{user}/Settings (the person, titled with their name, the owner only) and a node's ⋯ → Settings…; the mapping of every tab, the module API, the redirects that keep old links working, and the Inbox app every user gets
• In-App Extensions — packages that extend an app instead of being one: the Threads app's harnesses and model providers, the person app's signing authority; hostedIn + extensionSlot, the Store's Extensions shelf (Get or Buy there is the in-app purchase), and the PersonApp contribution gated on requireAddressAccess
• Domain Configuration Apps — configuration and secrets live in the app that owns their domain (AI, Databases, Sign-in, Email, Payments, Integrations), never as panels on the Deployment record page: the fleet apps on the control instance per deployment, the instance-scope tabs in each Admin app, what each reads, writes and gates, and how a change reaches the portal
• Invitation-Only Onboarding
• Service Identities — non-person principals (an integration, a bot, a CI job) with their own mw_ tokens: created by a global admin in the Admin partition, granted like any subject, audited under their own svc-… id, revoked with every token at once, and never a global admin
• Token Validation Hot Path — why an unavailable token check must never read as a sign-out (the MCP scheme's own 401 challenge), the per-replica cache and direct store read that keep validation off an unanswered hub hop, and readiness defined as "this replica can authenticate a token" (a sampled canary, three failures to leave rotation)
• The Redirect-Target Contract — every returnUrl sink validates local-only, so every source must mint local; a wrong source is refused rather than followed, which costs the whole flow and logs nothing
• Logon Actions — per-user work at logon, run as the user
• Unanchored Security Reads — why the permission fold reads mesh-wide, and why pinning it to the viewer's partition is a silent revocation-fails-open bug
• A Denial Is an Answer — a check on a hub with no evaluator grants Permission.All, and a refusal the mesh decided is rendered, never raised
• PublicRead and Denies — a Public/Anonymous deny under a PublicRead policy is honoured by the SQL read path and ignored by the C# evaluator; what that split exposes, and what each remedy costs
• In-Mesh Impersonation: who may act as the platform. These are the log-only guards on code the mesh compiles at runtime and on gate deliveries over the trusted gRPC port, the blast radius measured across the fleet, what the runtime guard cannot see (tail calls, delegates, hand-built contexts), and the options for closing it
• Who Owns a Partition's Access Shape
• Partition Ownership Resolution — the four create-path checks that ask whether a NodeType owns its partition, what one resolution costs for a type declared in mesh content, which of them share ONE view and which deliberately keeps its own, and how a nested instance of such a type is refused from the definition's durable row without activating the type's hub
• Protected Segments on a Public Partition — a partition that is public except for one inbox cannot be expressed with PublicRead: the C# evaluator and the SQL projection resolve a deeper deny under it differently, so the segment reads by exact path and is absent from every listing. The grant shape both folds agree on, why a read cap is a blackout rather than a gate, and why the boot heal may never retire a deny it could not have written
• Secure Development Framework — how production and development are separated when the platform is delivered into a client's own environment: a ringfenced client estate, a narrow interface, a consultant environment that never touches production data, and who approves what
• OWASP ZAP Scan — 3.0.0 (6 September 2026)
• OWASP ZAP Scan — Every Release

Threads, activities & AI

• Start here: Thread Operations
• End Vision — Jobs, Durable Streams and Document Ingestion — the architecture decided on 2026-10-03 (every activity a job in a queue on control, streams as nodes, triage that acts, document ingestion) and THE plan, with its executable e2e specs
• Thread Supervision — a round ends stamped, whatever failed; the one death the hub cannot cover is its own, and the supervisor, dispatch pool and Admin/Threads queue page that cover it
• Agent Task Collaboration — launch shared work only through start_collaboration; participant effort, harness, and model are creation-time settings, not follow-up-message overrides
• Thread Execution Streaming
• Activity Control Plane
• Activity Mirror Release Lifetime
• Activity Operations
• The Communication Hub — one participant registry, one message-type family (approvals, notifications, information requests) and the master-plus-copies fan-out that keeps every inbox query anchored to one schema
• Notifications
• Notification Retention — the platform's first data-retention pass, and why it is a logon action
• Agentic AI
• Script Execution
• Agent Framework Stores — mesh nodes behind Microsoft's abstractions
• Centralized speech — Whisper Swiss German as a container
• Email Ingestion, Channels and Notifications
• Event Subscriptions — the durable 'when THIS fires, run THAT' engine
• Foreign-Language Bridge (Python, Bun/Node) over gRPC
• Foreign-Language & Cross-Platform Integration
• Model Providers and BYO Credentials
• On-device voice — Whisper + Swiss German
• Voice model distribution — where a 547 MB CC BY-NC model may live
• Python Code Nodes
• Script Execution — Try It
• Sending Email

UI

• Start here: User Interface
• Blazor Data Binding
• Editor Field Keys — the node-content editor binds by JSON key, so a rename behind [JsonPropertyName] or a value the serializer drops discards what an admin set, silently and permanently
• Per-Tab Session State — a node is shared by every tab of one account, so "which page is this viewer on" and "navigate ME there" can never live on one
• Blazor Async
• Available Controls
• Node Page Provenance — the landing page carries "who wrote this, and when" by default; declining it is a sentence, not an absence
• Menus as Data — re-word a menu without a build
• The Menu Contribution Boundary — what may be data, what stays compiled
• Markdown Fence Extensions — the platform emits a marker, the clients hydrate it; a new fence is always a two-repo change
• Localization — the viewer's language, resolved explicitly, never from ambient culture
• Chrome and Content Language — ownership decides the language, and in-flow chrome minimises words
• Localized Refusals — a *Response.Error is a wire field and stays English; the activity transcript is the surface a viewer reads
• The Supplied Navigation Rail — a module supplies its own left-hand index, and core renders it in the order the module gave
• The Apps Home
• Content Favicon Rasterization
• Controls That Cannot Fail
• Catalog Action Identity — a retained click keeps its package when the catalog refreshes
• Link Previews
• Public Web Presence — one public host, a body in the first response, a sitemap that descends to every page a stranger may open
• A Zero-Root Sitemap Is an Assertion — the sitemap projected the tri-state anonymous gate onto a bool because omitting ONE undecidable page states nothing; omitting every root produces a 200 that says the deployment publishes nothing, so an empty surface nothing decided is now 503 and a partial one is still published
• Local-First Client & Bootstrap
• PDF Export — one browser, two fidelities
• UI Extensibility

Node types

• Start here: Adding a New Node Type
• A Typical NodeType — Budget, shipped beside the page: the files (NodeType json, Source/, Test/), the rules, how the bake and the gate compile it and EXECUTE its Tests area on every pull request, what green means, and the traps already paid for — with the live run embedded
• NodeType Supersession — a retired NodeType names its successor and how its content maps; an instance re-types itself when its hub next activates, which removes the migration script, converges LAZILY and therefore still needs a sweep, and requires the migrating activation to recycle itself
• Creatable Types — what may be created under a node: the one provider the Create form asks, what a parent NodeType restricts, what it extends, and why a parent that declares nothing must never narrow the menu
• Retiring a NodeType — the prune keeps the definition and deletes its sources
• Dangling NodeTypes — a node whose type resolves to nothing, and the two write paths that allowed it
• Node Type Compilation
• Closed Type Set — Mesh:ClosedTypeSet=true: a process activates only the NodeTypes its image registers in code; a database-defined type is refused at activation, never enumerated by the sweep, never adopted and never compiled — the property the dedicated control instance's readiness rests on
• Mount-Relative Source Queries — a cross-type shared=@… source reference is authored mount-relative, so the resolver asks for the mount-anchored spelling too; resolved verbatim under an import prefix it matched nothing while the type's own rebased query did, so the merged set stayed non-empty, no emptiness check could fire, and Roslyn reported a genuine-looking CS0246 about present content. Also: the four CS0246 shapes a reader has to tell apart from the source-discovery block
• Who Owns a NodeType Member — the repo owns the definition, the mesh owns the compile state, and the mask that encodes it was pinned in ONE direction: four runtime-state members were missing from it, one spelled outside the naming convention meant to catch them. The three guards, and why a member the mesh writes may still have to stay unmasked
• Compile Cache Input Freshness — verify the captured input before reusing a DLL that finished after a source edit
• Execute-Time Interlock — a build proven stale is never armed
• Emit Reference Capture — bounded, opt-in CI evidence for runtime compiler failures
• Graph / Compiler Layering — the four assemblies, the cycle, and the full-MVID size rule
• Toolchain Re-evaluation Lane — why a toolchain change stopped rebaking the world
• The Dependency Record Floor — a record's module entry says "I need at least X", not "I need exactly this build"; the MVID pin that could not converge because Roslyn hashes absolute source paths, the two-replica recompile ping-pong it produced, and the four things the floor deliberately does not relax
• Reading a Module's Stamp — reading ONE assembly-level attribute through reflection resolves the type of ALL of them, so a module with a private dependency killed the version read with a nine-frame reflection stack naming nothing; the metadata read that fixes it, and the measurement (23 of 24 declared modules carry a module attribute — MeshWeaver.Maps carries none) that says module-ness is a deployment fact, not a property of the bytes
• An Unloadable Build Is Never A Silent Default — a recorded build that does not LOAD in this process used to bind the mesh default configuration for the grain's whole life; the always-activated Hosting/PlatformBuilds hub that ran twenty hours without its inbox, fleet watch and build queue while its record read Ok, and the two hypotheses (a missing module, "a restart activates it") the measurements refuted
• A Reader Never Deletes A Shared Build — the loader deleted every store build older than the running framework DLL, so the first replica of each roll removed the current compatible builds from the shared volume; the per-roll recompile wave, the Store/Plugin "stuck AGAIN" boot window and the shared-volume "DIFFERENT build" refetch refusals were that one delete (residues found verifying #6161 and #6052, not their resolution)
• Producer Determinism of the Dependency Record — the same content must stamp the same record however the producer reached its bytes; the disk-cache hit that shipped a weaker guard, and why the digest is persisted beside the bytes rather than recomputed
• Rebake Waves — why a roll rebakes the world anyway, and what one rebake writes
• Source-Set Establishment — a resolved source set of ZERO is ambiguous, and only the type's own persisted snapshot tells "owns no Code" from "the discovery pass came back short"; the boot that resolved 91 fewer Code nodes than its neighbours and held a portal out of rotation for the startup probe's full three hours
• Missing Declared Sources — emptiness measured on the UNION is invisible for any type that also draws on a shared library; the NodeType whose own Source subtree was gone, reported itself as broken C#, and had the identical doomed compile taken again on every boot for four days
• Compiled Against A Platform The Instance Does Not Run — the source compile gate ran and PASSED: it compiles against the platform CI is about to ship, while the receiving instance is still on an older image. Six NodeTypes on the public instance stopped compiling forty minutes after that satellite merge; why the prebuilt was declined and Roslyn ran, and why the CS1061 localizes the fault to the platform surface without saying which side moved
• Install-Time Prebuilt Adoption — the only lane that serves a package installed AFTER boot, and the four answers its zero must keep apart because a silent non-adoption reads exactly like a successful one
• Adoption and the Sweep Count Different Things — the cold boot that adopted 78 prebuilt assemblies and then reported 5, with nothing wrong on the share: what each instrument counts, why the sweep could not see its own process's writes, and the node-version ordering that keeps the fix from becoming a stale serve
• An Adoption That Changes Nothing Writes Nothing — a prebuilt adoption that would leave the build unchanged uploads nothing and writes nothing, and two bundles offered for one NodeType converge instead of replacing each other: the version churn of #6038, its measurement, the rule, and how to check it holds
• A Census That Counts Must Name — the one past-RLS census counted a permanently-broken NodeType and dropped its path one call before publication, so its output read as clean; where the identity was lost, what a PUBLIC census may name (the partition, never the node title), and how to tell a fix that is merged from a fix that is running
• Dynamic Content Type Registration — a dynamic NodeType's content CLR type registers only as a side effect of one of its INSTANCES activating in this process, so a type this replica adopted rather than compiled is untypeable here with a perfectly usable assembly, a clean bake and a clean census; the instance-count discriminator, the disjoint instrument sets that falsify “the bundle was declined”, and why closing it from the degrade seam drives an adopted type into a compile and re-stamps a record the whole deployment shares
• Denied Is Not Absent — a Forbidden that a read discards comes back as the thing being MISSING, so the operator announces a node pool, an Ingress or a ConfigMap is absent when it was merely not permitted to look; the three answers a read owes its reader, why fixing the sites a report names does not sweep the defect, and the gate that makes the next one red
• Import Write Ordering — type before instance, and what a foreign type does
• Language Services
• Extensible Defaults
• Build coordination — the Build node protocol
• Build Identity Admission
• The Build Process — compile and test as a dependency cascade
• What a Pull Request Rebuilds
• The Build Server
• The Compile Program — State of Record
• Content-Type Registration
• Import-Side Content Degradation — the OTHER half: content degraded at IMPORT rather than at read. The /health content-types census cannot see it, because an import-degraded node holds a plain MarkdownContent whose $type resolves perfectly on every replica
• NodeType Catalogs (shipping instances of a NodeType)
• NodeType Release Redesign

Plugins & content delivery

• Start here: Plugins — node repos from git, no NuGet
• Plugin Manual — author · publish · install · own registry
• Plugin Registry — memex re-serves plugins over REST
• The Plugin Registry Index's Per-Request Cost — the authenticated bundle index evaluates entitlement per package and queries the mesh per request (12–19 s to first byte for 8.7 KB), and that one cost reads as two unrelated incidents because it meets a 10 s auth budget and an unbounded assembly stage; why widening a client budget moved the symptom instead of fixing it
• Webhook Inbox — external services deliver into /_Inbox
• Plugin Packaging — bundles, the framework identity, and the Release node that links a release to its assemblies per architecture
• Install Readability — the two doors an install can open, and the cover-grant deadlock detector
• A Module's Static Web Assets — a module's CSS/JS ride the bundle in their own folder and must land MODULE-RELATIVE beside the entry assembly; anything that copies only the closure loads perfectly and 404s every asset behind one Debug line
• Static Repo Import
• Module Sync Per Manifest Hash — an instance always syncs every module it has: each module is judged alone by the content hash in its manifest.lock (unchanged writes nothing, changed syncs, a declared platform floor above the running platform declines that one module), and the seal decides only adoption versus compile — never whether sources arrive
• Sources Sync on Push — a GitSync'd Space takes its sources on every push to the configured branch, at the pushed commit, never waiting for a green build; a periodic branch reconcile repairs a lost webhook; an import prunes only nodes the repository put there, never runtime state
• Source Retirement — when the repository deletes the whole folder a GitSync'd Space mirrors (a package renamed or removed), the sync retires the nodes that folder imported instead of refusing forever; how a deletion is told apart from a mistyped subdirectory, and what goes and what stays
• Declared, Not Derived — what the platform cannot run without (models, providers, agents, queues, policies, credential references) is declared in config or committed content; a boot-time importer never deletes a declared item because a module did not contribute it on this replica, and pruning removes only what the declaration dropped
• Adopt Then Sync, Per NodeType — the seal is a REPOSITORY fact and adoption is a per-NodeType one, so landing a Space on the sealed commit is not enough: an adopted type's sources wait for the bundle built from them, the rest of the Space imports, and a partially-held Space keeps the commit it genuinely holds
• The Sync-Ref Contract — an import of a repository whose bundles this instance runs lands on the commit they were baked from, whoever asked — a person's Update included since 2026-09-17; every other repository reads a commit CI proved, or a branch tip a person asked for; resolving the ref twice put sources no build had compiled onto two production portals for five hours
• Node Type Compilation
• The Platform Image's Closure — the image IS the reference set every satellite's modules compile against; the two invariants, and why every consumer used to discover them by failing to compile
• Compiled Against A Platform The Instance Does Not Run — the link gate measures a module's bytes against the platform ACTUALLY RUNNING; the source lane measures Code nodes against the platform it is about to ship, and judge-against-baseline — the arm that reads what the fleet runs today — is wired into core's promote, not into the satellite PR where the content half lands first
• Sealed Publication Reads — a publication is unreadable for ~90s per target per publish and the plugins prefix has two writers; the three answers (404/503/412), the generation that pins one instance, and the mix no reader can detect
• Sealed Publication Generations — the layout that makes that mix UNREPRESENTABLE (a directory per publication plus a pointer swapped last), the reader contract, the retention rule, and the ordered migration that avoids a new-writer/old-writer half-state
• Install Completeness — what an install RECORD declares landed, compared against what is actually in the mesh; the comparison nothing made until #3485, and why only one of its five verdicts is a pass
• Declared Is Not Landed — the three exits an install can take and the one that never looked at the mesh: an UPDATE fetched the manifest diff alone, so a node lost after a previous install survived every subsequent update, each reporting success. The measured case, the batched presence read that fixes it, and the sweep hazard that made the damage read smaller than it was
• One Partition, One Bookkeeping — a partition written by BOTH a GitSync source and the registry installer keeps two independent records of one mesh; the seal reconcile rewrites the content and the next registry delta is computed against a record that stopped describing it, so Store on memex became a mix of 1.10.3 and 1.11.1 and Store/Catalog parked on CS1061. The invariant, why resetting the record or diffing the mesh both produce a ping-pong, and the two gates
• CI Content Bake
• The In-Mesh Warning Standard — in-mesh C# is the only C# no -warnaserror build ever sees, and the bake was discarding its warnings too; the two shrink-only ratchets (real warnings, and CS1591 on its own), why the RUNTIME compile must stay lenient — a parked NodeType refuses readiness and stalls a rollout — the three codes the platform itself was emitting into content it does not own (850 raw occurrences → 375), and the observe-only default that lets a repo adopt without going red
• A Gate Must Compile the Same Program — the pre-push NodeType gate compiled each source file on its own while the mesh concatenates them into one unit, so a nullable-context directive in the first file (in force in the last) reached nothing and the gate was blind to diagnostics the bake then filed under the NodeType's name with no file and no line; what the unit boundary decides, the culture-sensitive StartsWith that a naive reproduction gets wrong, and the parity test that caught it
• Bundle Delivery Stages — the four independent stages between a merge and a portal serving prebuilt bytes (write · compose · select · deliver), which of #3461 / #3732 / #3768 / #3583 owns each, the instrument that answers for each — and why a reading taken at one stage is not evidence about another
• Framework Identity Churn — the identity moves on every core COMMIT, not on every content change (43 merges, 5 touching the full-MVID set, ≥18 identities in 24h); the commit sha compiled into AssemblyInformationalVersion is why, the falsification test that refuted the local fix (0 of 22 control, 22 of 22 and 21 of 21), and the four costed options with every runner-hour labelled as arithmetic
• Prebuilt Bundle Retention — the sweep that prunes what CI bakes: where it is registered (and why "zero callers" was measured twice and wrong both times), the deletion default that is true in code and false in the chart, the report that names its denominator, and the pinned satellite gate the protected set cannot see
• Deploying a plugin change — merging is not shipping
• Module Activation Head Ownership — which generation a deployment runs is a decision every replica writes to ONE shared file and replaces unconditionally; why the same-content case is already benign, why the regression propagates into the proposed module set rather than being bounded by it, why routing the write to an owning hub is a cycle (boot reads the record before the mesh exists), and what deriving the head costs measured rather than estimated
• Module Adoption Policy
• Publishing A File On A Shared Volume — a name readers watch must appear holding the whole file or not appear at all, and File.Move(…, overwrite: false) does not promise that: on a volume without hard links (Azure Files) its failed rename COPIES into the final name, which is then incomplete and exclusively locked for the length of the copy (measured: 21,573 sharing violations, 52,619 incomplete reads, ZERO "absent"). The primitive that replaces it, what it refuses rather than copies, and what is still not atomic
• Module Build Architecture
• Module Closure Accounting
• The Module Identity Anchor
• Two Identity Schemes, One Comparison — a bundle states a g<sha> commit identity and a portal resolves an s<hash> surface identity, so the #4161 discriminator answered "different" for every pair in the fleet: measured on the control instance, eight modules declined on every boot and reported as "a restart activates them" across a restart that could not clear one of them
• Module-Owned Siblings Ride
• The Module Platform Link Gate
• The Platform Compatibility Ladder — platform builds are backwards compatible within a major and a compatibility epoch, so plugin bytes built on N run unchanged on N+1; the floor/ceiling/epoch that bound it, the machine-checked break declaration, the API-break catalogue (static verdict vs what really happens at run time), the check that blocks every platform pull request on the deployed plugin set, and what no surface check can see
• The Module Update Ladder — Rule Table — the ladder written as rows a test can check (platform fixed while a module moves, platform rolls under unchanged modules, store copy against image copy, prebuilt adoption, seals, source sync, the control instance, the 2026-10-05 incident), each row tied to its policy, the code that decides it and the test that holds it, plus the ordered P/M matrix every instance is walked through
• Rolling-Update Build Tolerance — why a rolling platform update recompiled Store/Plugin ten times in four minutes and blanked every instance behind "build did not settle within 30s" (two generations, one record, one framework identity per record), and the rules that let an instance render on the last build its process can load instead
• The Module Publication Gate — a bundle used to reach the live registry from inside its own pack leg, before the sibling suites, the portal-host shards, the compile-check and the Tests-area gate had reported; the hand-over moved downstream of the full source verdict, and what it refuses (failed, skipped, cancelled, missing, foreign-lane, substituted)
• Live Module Update — a module update goes live in the running process by default: every module in its own collectible load context, the old generation swapped out and unloaded; a restart only when a module declares restartRequired or the live swap fails (policy module-live-update-default)
• Module Generation Substitution — Assembly.LoadFrom does not promise to load the path it is handed: a byte-identical copy the load context already holds is returned instead, silently, so the loader recorded the generation it ASKED for while the process ran another; the three answers that replace two
• Module Set Convergence
• Module Versioning
• Modules
• Required Module Authority — Modules:Required is an array and configuration merges arrays BY INDEX, so a record's list is an overlay and not a statement: a shorter list leaves the image's tail required and an EMPTY list requires MORE; the scalar claim that lets a record say "these and only these", why it is opt-in, and the two instruments that make the gap visible
• Package Mark Inheritance
• Pin-Boundary Contracts
• Platform and content — two layers, two cadences
• Platform Build Identity
• The Platform-Shipped Witness
• The Plugin Build Contract
• Plugin Bundles in the Registry
• The Registry Listing Cache — GET /api/plugins re-read the whole repository per request and blew its 30 s budget ~60×/day; what is cached is the SOURCE snapshot, never the response, which is what keeps the fix from becoming a disclosure — and the read itself now transfers only the manifests it parses (47.8 MB / 13 s becomes 1.3 MB / 3.3 s) instead of the whole repository
• The Bundle Transfer Budget — a module adopt's 120 s attempt used to cover the whole archive download, so the budget measured size ÷ throughput instead of whether the registry was answering; 18 adopts failed at exactly the outer 3-minute bound recording neither bytes nor elapsed time, which is why they could not be explained
• Plugin Publication Provenance — the signed publication callback names the CONTENT commit that was built and the platform version read from the selected portal image, never the calling workflow's commit or event; core CD building Plugins used to announce a core sha as a Plugins commit
• Plugin Update on Green Build

Reliability & wedges

• Start here: Error Propagation & Wedges — drive wedges to 0
• Which Kind of Silence — a quiet log has four causes with four owners; the liveness heartbeat that separates them, and the CPU sample and page snapshot that agreed with the wrong one
• Action-Block Wedge Prevention
• Riding Out a ShuttingDown Address — the one transient NACK, and the two axes a ride-out must bound separately
• Hub Initialization Failure
• Orleans Stream Pub-Sub Durability — a publish with no subscriber succeeds, so a cross-silo reply can vanish with nothing logged
• Durable Streams Are Mesh Nodes — the design that retires the memory stream without a provider
• Durable Streams — a stream of saved mesh nodes consumed inside a hub: ordered appends, a lease that is the subscription, an acknowledged checkpoint, takeover, orphan and wake
• The Pod-Hub Claim Must Be Re-Asserted — a claim asserted once into a directory that is re-partitioned on every membership change is lost silently, and forever
• Oversized Delivery Refusal — a message too large for its transport destroys the connection carrying it; refuse at the producer, never raise the limit
• Content Sync Visibility — a Space whose assets the transport refuses says so, on the Space itself, naming the file, its size and the limit
• Out-of-Band Content Transfer — a content file too large for one delivery travels through the content store behind a content-addressed handle, never on the message
• An Unreachable Store Is Not a Refusal — one classification, three consumers; reporting an availability failure as a verdict is how a retried create becomes a duplicate
• A Name That Does Not Resolve Is Not Transient — the default pipeline retried a hostname that does not exist three times and logged each attempt at Error, so a URL in somebody's data manufactured a platform incident; the one socket error that is permanent, why the breaker must not count it either, and the control that keeps a nameserver hiccup retryable
• Every HttpClient Names Its Pipeline — Source: '-standard//…' was the shared defaults pipeline every non-re-registered client rode under one empty instance and one circuit breaker, not an unnamed client; the pipeline instance is now the client's own name, and the timeouts behind it were the self-update hand-over meeting a restarting control instance
• A Departed Silo Is Not a Delivery Defect — two incidents (191 and 3,959 occurrences) read as two defects for four weeks and are ONE root; which predicate sees which of the four rejection shapes, why the cure is the classifier and never a retry, and why an incident fingerprinted on a dependency's logger counts attempts rather than verdicts
• A Timed-Out Delivery Is Still Held by the Callee — a response timeout is a caller-side give-up timer, so the six-retry ladder sized for an instant rejection re-sent seven copies of every slow delivery and held a dispatch slot for 3 m 40 s; the three-predicate ladder that separates "is this transient" from "may we send it again", and the one caller that keeps the wider answer because it is idempotent
• The Subsystem in the Title Is Not the Cause — six tickets named two Orleans subsystems and grouping them on that was wrong three times; the three questions to ask per ticket, the pod-hub address suffix that is a process id rather than an operation id (which moved one investigation to the other side of the call), the two memory-streams tickets whose live traffic has different owners, and the precondition for folding a symptom into its root
• Reading a Routing Saturation Report — the [ROUTE] back-pressure line is a gauge over three different facts, and for most of the tickets it produces the causal arrow runs the other way; which number answers which question, the two opposite pool states that printed the same number because the report showed the one gauge blind to the cause it named, and the oldest-leg age that separates load from a leaked slot in a single sample
• The Compile-State Satellite Is Retired — a phase-1 dual-write nobody ever read cost one node-operation round trip per NodeType activation and one grain activation per NodeType per state move; measured as the destination of boot-time placement timeouts, stalled activations and routing back-pressure legs after the compile lane moved off the ThreadPool, and removed rather than batched
• Compiling Off the ThreadPool — a NodeType compile held the silo's shared ThreadPool twice over (its async leaf ran Roslyn on a pool worker, and ConcurrentBuild fanned each emit out onto the pool); the probe that measured it, the fix, and the roll-time routing starvation it bears on
• Blocking Leaves Off the ThreadPool — every IIoPool's blocking leaves borrowed ThreadPool workers under caps up to 256, and a kernel cell's compile ran on one; what the production stall readings (#5388) say was GC and what was blocking, and the fix
• A Bulk Create Compensates Per Node — every row is durable before any post-creation handler runs, so one critical failure left the failed node AND every node after it, whose handlers never ran and which nothing can tell apart from a success; what the rollback removes, why it walks backwards, and the measured reason the stop is a fault and not a Take(1)
• Undetermined Is Not No — a read that did not answer is a THIRD state; the second door that shared the first door's failure domain, and the rule for what a gate does with "I could not determine"
• Reading a Silo Eviction — a heartbeat newer than the suspect votes is not proof the silo was healthy; the control arm that tells a correct eviction from a false positive
• Dead-Circuit Fan-Out Storm — a closed tab's owner pushed to the corpse for 46 minutes because the only verdict the eviction acts on could not be said; the release tombstone that says it
• Bake Seal — NodeOps Saturation — the mesh's ONE node-CRUD hub stops draining under a bulk burst, and every consumer then reports its own bound
• Live Query Re-query Cost — every write re-walked the security fold's permanent live queries, so one write cost more the larger the mesh; a live raw query now ignores a change whose old and new node types it can never return
• The /api/content 503 — three causes with different fixes, why the third wears the first's signature, and the black-box discriminator that needs no log line
• Refused Replies During Teardown — every failure route answers the SENDER, which for a reply is the responder; the answer the caller is parked on is dropped with nobody told
• Refusing a Lost User Action — a click whose stream is gone is refused out loud instead of dropped as churn; why "deliver it anyway" is not implementable as stated
• Input From the Subscriber — a layout area is rendered once per subscriber, and its stream accepts a click, a blur, a dialog dismissal or an edited value only from the identity it was subscribed under; anything else is refused with an answer
• Guards and Unknown States
• Mesh Admission
• The Bake Gate Only Stalls a Roll — a type that never built was read as a regression, a refused pod could not record the failure, and a restarted pod of the serving image refused itself until nothing served; a regression now needs a working build from another, older image, and an image that has served here never refuses itself
• Disposed Scopes and Dying Hubs — one symptom, five roots — every ObjectDisposedException on an Autofac LifetimeScope looks identical in a log and comes from one of five unrelated roots; the discriminators, which are fixed, and the swept inventory of deferred resolve sites
• Enumerating from a Survivor — a recycle derived its work THROUGH the hub it was tearing down, so only the synchronous prologue ran while that hub was whole and every asynchronous leg after it resolved from a closed scope; the cascade recycled 0 addresses while reading as its own success. Hoisting the resolve — the correction for this family — does NOT fix it when the service is scoped PER HUB, which is the companion clause to R3
• A Stale Index Row Confirms Itself — a torn-down partition leaves index rows behind, and the listing that would disprove one is answered by the same index, so absence is unaskable rather than merely unproven; the storage providers are the only independent witness, the answer is worth nothing unless it is asked BEFORE the work, and fail-open is the only safe direction
• Mesh Lifecycle — Build Up & Tear Down
• Ordered Route Channels — the FIFO Key Is (Destination, Stream) — a stream-routed address is a multiplexer, so keying the router's ordering FIFO on the destination serialised a whole process's data-sync traffic into one lane with one in-flight cross-silo grain call; 62 of 64 dispatch slots queued behind one cache hub for 1.5 h while the peer pod's local lane read as load, and the identity that separates the streams was already stamped on the envelope
• Pod-Hub Delivery — the Transport Swap and its Roll Plan
• The Portal Heap Is Hubs
• SignalR Mesh Participant — joining the mesh over a WebSocket
• Teardown Layers — work finishes, nothing is forced
• Teardown Verdicts Are Causal, Not Timed

Testing & debugging

• Start here: Writing Tests
• Negative Controls — a pin is only a pin if it fails against the defect
• Fault-Injection Harness — deterministic, releasable faults (silo kill / drain / linger, flush hold, NotFound window, change-feed and first-frame holds, webhook loss) and one isolated regression per production incident, each with its negative control
• Reactive Test Assertions
• Test State Isolation
• Disposable-mesh e2e
• Debugging Message Flow
• Debugging Disposal & Leaks
• Departed Platform Assemblies — an assembly that leaves the platform for a module breaks every OTHER module that binds it, at LOAD time and invisibly to every compile gate; why "those are the platform" is one answer per host
• Detached Response Continuations — why a hub.Observe(...) continuation runs on the RESPONDING hub's action block, what that cost on the mesh's one node-CRUD hub, and the six invariants that make the hop an opt-in rather than the default
• Reading a Disposal Stall Verdict — what each field of the disposal snapshot actually measures, the three that were read as evidence while measuring nothing, and the verdict hole that sent 47 reports to children that were not the problem
• A Failure Report Answers Its Own Instruction — a report that says "find why this hub disposed" while holding the answer, and an outstanding-work field that rendered "not measured" identically to "none"
• A Fault Does Not State the Verdict — a fault logged at Error claimed the outcome a later branch decides, so once a recovery path was added every benign blip filed a red line asserting held rollouts; why the watcher's Error-and-Critical capture rule makes that claim the permanent record, and why the issue could never stay closed
• One Log Site, Many Terminals — an incident fingerprint keys on the message, so it SPLITS one log site across tickets and FOLDS unrelated roots into one; how to group a cluster of timeout tickets on the innermost TERMINAL before fixing any of them, why a count whose log statement precedes the recovery branch establishes neither breakage nor a fix, and the three reasons a frozen lastSeen is not the fix working
• Ambient Test-Host Hangs — what decides whether a killed test host can be diagnosed at all, and the readings of it already falsified
• In-Mesh Tests and the Seal — a Tests area no required context executes is a latent trunk red the seal detonates fleet-wide; how to measure a gate before requiring it
• Cancel and Join Are Two Questions — a deadline that asks work to stop and a deadline that waits for it to have stopped must not share one clock
• Collection-Scoped Test Fixtures
• Debugging Native Crashes (core dumps)
• Collectible Thread-Static Handle Reuse (CoreCLR) — a thread exit frees a slot in a LIVE collectible context's handle table when a thread-static index was reused after an unload; sighting #18's dangling static base, reproduced deterministically
• Peer Death Is Dated by Its Exception Class — a frozen silo and a dead silo raise different Orleans exceptions, so the changeover between them timestamps the crash; how one pod's SIGSEGV became three issues, and how to get a dead pod's own account out of Loki
• Reading the Memory Watchdog — a step with no mesh class active is a plain test class, a ramp across mesh classes is retention; the guard states what it measured, never a cause
• Debugging Postgres in Prod / Test
• Decentralised Tests
• Gate Content Assets
• In-Mesh Build and Test
• Orleans Test Routing Pattern
• Reading CI Signals
• The Canonical Node-Test Harness — run-node-tests.py has one home and the satellites launch it; how three vendored copies broke silently and differently the day the compile model changed, and why the one-line revert is measurably not the fix
• Which Attempt an Artefact Belongs To — a run holds every attempt's artefacts and the API names no attempt; how the required check consolidated attempt 1 over attempt 2's green and could not be re-run to green
• Workflow Permission Pairing — a job-level permissions: in a shared lane is a requirement on every caller; an unpaired one is startup_failure with zero jobs

Deployment & ops

• Start here: Deployment (the router)
• AKS
• Planning a Database Migration — the process (policy db-migration-planned): expand-only migrations, the Db-migration: declaration and its rehearsal, ExpectedDbVersion on the release marker, and every roll path migrating first or refusing — with the operator interlock that does not depend on the build being rolled
• Database Migration Procedure — the schema moves before the image, every roll; the 2026-09-03 wedge behind a 200, the recovery, and why a migration deadlocks under load
• Container Apps
• Hybrid delivery — GitHub for code, Azure DevOps for the deploy — for organisations whose change management lets ONLY an Azure DevOps pipeline touch the cluster: the GitHub App that merges to main is the App that triggers the deploy, one Azure Pipelines run per SEALED image set (never per push), the record selects the executor, and the pipeline ends with one signed report — same record, same action, a different executor
• In-cluster databases — each instance's PostgreSQL as its own Helm release: a CloudNativePG Cluster on a dedicated db node pool, primary and standby in two zones; why neither the chart's bundled Postgres nor the shared Flexible Server serves a client instance
• Local Dev Workflow
• Onboarding a New Environment
• Unclaimed Control-Plane Requests — an InstanceAction at version 1 with an empty log means "queued", "nobody is listening" and "the operator died holding it" in the same bytes; the 2026-09-10 measurement, the Ops/Status staleness that DOES discriminate, and the acceptance signal that does not exist
• Release & Self-Update Strategy
• Release Channels — a channel is a named, moving pointer to an immutable release; latest is derived and only stable is promoted, and a channel names what to SELECT while the selection always resolves to an immutable id that is what gets pinned, recorded and run
• Release Support Policy
• Released Artifact Retention — retain artifacts for at least 30 days, supported releases for their support lifetime, and every artifact still needed by a published set or consumer
• Platform Versioning (SemVer) — continuous builds move from 3.0.0-ci.<run> to plain 3.<minor>.<run>, where the patch is the same CD run number, so both notations share one lineage; 3-latest is a seed pointer and never a record value; the inventory of every producer and parser across core, Plugins and Memex; and the readers → records → minter migration order
• Self-Update Target Selection — candidates are ranked by the CD run number, not the version string; a mislabelled line outranked every sealed set for ever, and an install on a withdrawn tag could never see anything newer
• The Self-Update Registry Credential — which plugin-registry key may be presented to a container registry: a DECLARED pairing, never host equality or name resemblance; an absent declaration refuses
• Package Uninstall — one request in two phases: retire the module, close the hubs, remove the install record, block re-install and preview what would be destroyed; drop the partition storage and its registry record only on the requester's exact confirmation; refuse a shared partition or one holding user data
• Module Reload — one request, "reload module M on this instance": resolve the newest compatible published version by its declared floor (never a seal), land it, activate it live or with exactly one automatic restart, and report on the request node what was found, what landed, how it activated and what every replica loaded
• Instance Reboot — one operation, "reboot this instance": sync every module source, land every module's newest compatible version, pick the newest admitted image, ONE roll or restart, verify every booted process (health plus a thread-start smoke check); a person's call is the signature, and the instance's own watchdog may trigger it on an explicit wedge predicate, rate-limited and alarmed
• Self-Update on the Control Lane — detection stays on the instance, the apply is one signed event to the control instance, the chart's one declaration binds the self-patch Role to the poller's intent; no portal holds a credential that changes the cluster
• Self-Update Announcement Key — a deployment that must not hold the fleet inbox secret announces its own self-update with a key of its own; the key causes nothing but that record's self-update events, and a record that declares one is no longer announceable with the fleet secret. The deployment's administrator generates it under /Admin/Settings/ControlLane, and the control instance registers it
• Instance Secrets — secrets a global administrator enters in the portal: stored encrypted in the instance's own mesh, used live with no restart, never read back; the write-only secret control, the fingerprint rule, and the slots that limit which settings the portal may set
• Control Lane — the signed control→instance lane: the control instance runs an approved Recycle or DeleteSpace INSIDE another instance's mesh; the target verifies signature, expiry and single use, computes its own plan, refuses any plan the approval did not bind, and reports every step back signed
• The Continuous Delivery Contract — all-or-nothing publication; verify the image, never the tick
• Why the Fleet Stopped Rolling Itself — the September 2026 measurement: a frozen heldReason is history, not a hold; a policy record that lost its own policy; "waiting for an approval" is not a freeze; and the seal that a cross-repo pair skipped
• Reading a Bake Publication Receipt — the four target outcomes and what each licenses; the one that had no word rendered "already everywhere" as "reached nothing", and two readers acted on it
• CD Reconciles the Plugins Seal — a set seals on its trio alone, so it can seal with no plugins publication for its framework identity; why the reconciler REPAIRS that rather than the seal forbidding it, and the three probe answers of which only one licenses a re-attempt
• The Self-Update Schema Wall — every schema-bumping release is un-takeable by self-update, the stall is invisible, and a promoted tag is not a deployable tag
• Bake Identity Mismatch — why a green CD can publish a bake no portal adopts, and the one rule that keeps two images of one commit on one address
• A Set That Contradicts Itself — two CD runs of one pair left a 9985 tester and floor next to a 9984 portal; the gates that keep a set's halves on one build
• Release Availability Gates — one predicate; never roll or build into a release a package cannot survive
• Deploying Across Platform Versions — the ladder: platform and plugins roll separately; a seal gates a roll only behind a declared break
• Adding a Data Sync Needs a Global Admin — a sync is a standing grant, not a one-off action: it keeps moving data indefinitely under an identity nobody re-examines, so the review at creation is the only one there will be. The trigger list, what is explicitly NOT covered, why a credential request instead of a proposal is the anti-pattern, and the one finding that blocks rather than being filed
• Policy Not Prose — never hard-code a decision's date or author into source, a comment, an XML doc comment or doc prose; a policy is a record with a value and an in-force date and everything else cites its id. The register, what counts as evidence rather than a policy marker (and so stays), the review finding, and why it is forward-only with no backward migration
• Issue Taxonomy and the Release Readiness Gate — four axes on every issue, and the one that is a gate: sev:B and sev:H must both be zero to cut a release; why only bugs carry a severity, why a zero is only trustworthy when its query's coverage is, and why the ledger stays in GitHub rather than being mirrored
• Combo Gate Wiring — the roll consults the combo verdict; Red refuses, and "could not find out" is neither
• Roll Selection — completeness as a SELECTION criterion: pick the latest release that ships all of an environment's plugins, refuse an empty denominator, and never roll backwards
• Release Process
• NuGet Package Retirement — two packages survive (the Aspire integration and the dotnet new template) and the other forty-three are unlisted; what unlisting does and does not break, the derived retirement sweep, and the ground rule that startup dependencies become Aspire options rather than new packages
• Repository Dependency Direction — the platform never depends on a plugin repo; the inventory of every edge that still does
• The Cross-Repo Pair Gate — a removal here reds a plugin repo's trunk hours later; the deleting half lands LAST; and the REQUIRED dependent-suites verdict runs the Plugins suites a diff can reach against the candidate (a measured break merges only when declared)
• One Promotion Gate — the one PR-side wait is the affected-only Plugins verdict (policy dependent-suites-affected-gate); always build the newest green core with the newest Plugins, and arm the fleet on the platform verdict (the arming mechanics: cursor, completion marker, resume, base)
• Platform and Module Deploy — two separate deliveries: every platform build goes to control first and the fleet only after control runs it; modules publish and auto-update on their own lanes; the alarm that keeps control on the latest build; what was measured and removed
• Paired Change Sets — pair a core and a Plugins pull request so the two are built and tested together before either merges
• Platform Script Resolution — a repo runs the platform's gate scripts, never a copy; the local runner must resolve the ref the LANE resolves, which is per-script, so a loader copied from another repo refuses on every call (one repo demands one lane sha, another pins four and the pin gate calls that consistent)
• The CI REST Budget — every REST call a satellite's CI makes with github.token spends one per-repository budget (1,000/hour); the lanes' per-file script fetches were the largest consumer, so they read core over git now, the stage-advance listener stopped answering every reply twice, and each job prints the budget it has left
• A Stale Run Listing Is Not a Broken Main — GitHub serves the platform resolver a weeks-old page of workflow runs, per call and with no error (three measured); why only one of the two listings can be checked, why neither retries, and the refusal wording that sent two readers at a main that was fine
• Keeping the Platform Source Pin Current — a satellite pins WHICH core commit its src/ compiles against, the image set had a mover and the source ref had none, and a bump PR opened with GITHUB_TOKEN is a PR no CI ever runs
• Transitional Allow Entries — an allow entry is written for ONE merge and expires with it by mechanism; the instruction that was ignored once cost every C#-touching PR in the fleet ~40 minutes of red
• Closing Keywords and Issue State — GitHub binds a closing keyword to the one reference after it and reads nothing else, so "Does not close #N" closes #N; the three misfires of one day (a disclaimer, a partial claim, and a docs PR narrating the first two), why a severity label may not be closed by a merge at all, the three-check gate, the per-issue escape that keeps it from being a wall, and the four blind spots it keeps
• Pinned Image Retention — registry retention deletes what CI pins, and republishing frequency is what destroys a pin rather than what protects it; the guard that names a dead pin, and the retention design that stops the deletion
• Artifact Retention Interlock — the one mechanism behind the four retention issues: cleanup may delete only what a COMPLETE and FRESH consumer inventory shows to be unreferenced. Three axes (the third asks each installation what it is RUNNING, because a committed pin is a proxy that drifts), the denominator every run must state, the TAG lock a manifest lock does not provide, and the instrument control that replaced an assertion about the fleet
• CI Artifact Storage — where CI's big build outputs live: the measured $260/month GitHub Actions storage bill and the org budget that decides whether a private repo can upload at all, the ONE artifact family that is read across runs (and why most of its bytes are duplicates of themselves), the 66.8 GB uploaded for a reader that was never built — and the stale checkout that made "nothing reads this" wrong, why an EXPIRED artifact is still billed and still deletable, the object-store seam and the degrade rule that keeps the public repo working, and the grants and the one variable the migration still needs
• PR Artifacts on Our Infrastructure — moving PR compute to our runners did not move its artifact BYTES: the named-artifact transport (store: on the shared upload/download actions, a declared store that cannot be used being an error and never a quiet fall back to GitHub), the partial-rerun selection rule, and the four rollout gates that separate "the transport exists" from "a private PR uses it" — including why queue admission is not a native build worker
• Fleet Registry Retention — the same question asked of cr.meshweaver.cloud, the fleet's OWN registry and the default for new instances: what deletes today (nothing — enumerated, with the one row that is a maintainer read), and why a registry with NO LOCK needs a stricter rule than the ACR rather than the same one, because there the derivation IS the whole safety margin
• The Image Tag Contract — which image tags the promotion actually publishes, why the portal has no latest, and the two-writer history of the one that had no producer at all: retired lane, then retention, and every check green throughout
• Pin Set Consistency — every pinned digest EXISTING is not every pinned digest naming the same BUILD; the invariants that red a half-moved set, three written deliberately weaker than the obvious version, and the falsification that found the vacuity trap inside the gate itself
• Duplicate Keys in Workflow YAML — a duplicate mapping key is accepted silently and the LAST one wins, so a pin can move in the diff and not in the job; the near-miss, why every existing gate was blind, and the guard that names the file, the key and both lines at the first job
• Two Files, One Node Path — the importers derive a node's path from the file's location, so Foo.md + Foo.json (or A.md + A/index.json) is ONE node and the import keeps whichever it wrote last; the rule the validate lane's guard mirrors, and the one pair the file-system adapter merges on purpose
• Image Pair Skew — a promoted image pairs a core commit with a Plugins head resolved hours later; each half green, the pair never run (the 2026-09-03 sign-in outage)
• The CD Ledger Records a Failure, Not a Cadence — 109 of 109 incomplete image set alarms recorded a delivery that had not failed: the pair tag that cannot be satisfied because Plugins merges faster than a publish takes, the minutes before a publisher even exists, the registry fingerprint that separates an attempt from an absence — and the rebuild-cadence decision this deliberately leaves open
• The Merge Queue — OFF on core main; its design, and a steward that re-queues an ejected PR on evidence and never re-runs
• Review Findings Answered — a pull request reads red until the automatic review has landed and each thread it opened has a person's reply; the reviewer's two logins, the quota refusal posted as a review, and the maintainer-only waiver
• Staged Pull Request Pipeline — the expensive suites on a fresh merge with the current main beside the review, then arming only when the review is answered and every required check is green (policy suites-parallel-with-review): event-driven advance, three loud releases, and the measurements
• Fresh Merge Under Test — every suite job tests the head merged onto the base's CURRENT tip, computed in the job and reported, never the push-time refs/pull/N/merge (policy suites-test-fresh-merge)
• Carving Projects Out Of Core — what a SOURCE move costs and what it does not
• Red-Log Watching & Ticketing — every fail:/crit: becomes exactly one triaged issue
• Log Entries Are a Query Result, Not a Feed — Hosting/LogEntry is the output of one Logs action, so an absence in it is evidence of nothing; the denominator printed on every row, the level that lives on a different node, and how to ask for a line that carries an answer
• Telling a Stalled Pipeline From a Dead One — the incident store carries three clocks and lastModified is triage bookkeeping, not ingestion; which query reaches detection and which reaches delivery (and the three that error), the normal-lag-then-silence signature that means dead rather than behind, and why a missing pipeline self-finding is evidence of death rather than health
• Verifying Chart Values — a key can be set, reach the render, and still not be read; why the obvious gate was vacuous for the one component it existed to guard, and the binary check that closes it (the drain that erased every namespace's log history)
• Measuring a Live Portal Read-Only — /health first (public, past RLS, a different replica each call), the incident store, the four break-glass instruments, and why an absence needs a coverage fact before it counts as evidence
• Chart Ownership and the Runner Pool — why the chart's gate is here, what a relocation must carry, and why path-filtering it is unsafe
• Sharding the Node-Repo Gate — a cap cut reports as cancelled, so the fan-out that removes it, and the fold that keeps ONE required context and ONE gate log
• Applying Is Not Rolling Out — helm applies, the caller observes; the fixed fifteen-minute --atomic --wait that reverted a correct upgrade mid-startup-gate, and why a bigger timeout only moves the cliff
• Probe Semantics — readiness, liveness and startup ask three different questions with three different remedies; why they get three paths and three tags
• A Probe Must Answer Inside Its Own Timeout — /health reached 8-10 s against the 5 s the startup probe waits, so a healthy replica could never leave startup and was killed at its 3 h budget; why a startup timeout is the unrecoverable one, why a HEALTHY slow check is the one nothing could name, and the timing line the endpoint now publishes
• What a Synthetic Probe May Assert — a probe naming one deployment's installed content is broken by construction; the platform floor, the negative control that tells "absent" from "down", and reading the target's own declaration
• Why a GC-Bound Pod Stays in Rotation — the GC's hard limit sits below the container limit, so a portal short of memory is defended rather than restarted
• Runner Pools and Dispatch Queues — two runner pools sharing all capacity; priority from the control instance's CI queue (express > trunk > gate > pr), dispatched in tier order with express headroom, re-orderable through MCP, failing open
• Self-hosted CI runners on AKS — ARC beside the portals on one pool; three brakes, a negative priority class, and the reserve arithmetic that decides the cap
• Candidate Release Protocol
• Chart Drift — what a deploy actually does
• Rendering a chart you are not allowed to fully configure — 39 runs, 39 failures, zero verdicts: the check may hold two of the deploy's three value sources and the chart correctly refuses that subset; the placeholder that unblocks the render, the two-render proof that no compared object depends on it, and the bake gate that was off on both production namespaces the moment a verdict finally appeared
• Configuring an instance from Aspire
• The Dependabot Secret Store
• GitHub App Credentials — meshweaver-cloud writes to its own repo; every cross-repo READ mints from the read-only fleet-reader
• Deployment env layers — what a record must be able to hold
• DeploymentInventory
• Deployment Options (AKS)
• Environment Composition
• Feature Flags
• First-Run Setup
• First-run setup on a PROVISIONED instance — a fleet-provisioned instance boots configured and unadministerable; the wizard keys on "no storage" and the onboarding gate on "no admin grant", so neither fires
• Image Cleanup
• Instance Identity and Setup
• Instance Lifecycle — State of Record
• Instances
• Local memex on Colima k3s (Mac)
• Mac local stack — on-device AI + local observability (M-series)
• Memex Cloud Deployment
• Merge Queue Mechanics
• The fleet's ONE CI process — the mechanism lives here as a lane or a shared script; a satellite carries a thin caller (verdict adoption, the lock resolver, the cancellation rule)
• Operating from the portal, not the cluster
• Operator Credentials Travel by Name — the operator's only environment is a ConfigMap, so a step is handed a Key Vault object's NAME and reads the value with the identity it already runs as; why every database action failed at step 1 (Memex#132), why the CSI class built for it was consumed by nobody, and the one contract the Postgres steps now share with hosting-kv-ensure on both executors
• The Payment Provider Contract
• Secrets: Write-Only Entry, Split Identities — nobody touches Key Vault by hand: a write-only GUI per owning app, ONE governed writer identity that can list/set/delete but never read a value, a reader identity (the pods) that can only get, the write-time fingerprint tag, and the CI guard against new hand-run vault commands
• Pre-Boot Service Substitution
• Project Templates
• Reading a Recurrence Reopen — a bot reopen asserts two things and both fail independently; the 2026-09-17 wave measured, and the honest triple behind core's count
• Reopening on Image Provenance — the third reopen predicate: an occurrence counts only if it came from an image whose commit contains the fix, and why a staleness window would close live defects
• Registry-key rotation — two phases, at the registry that holds the instance
• The Release Event Bus
• Release Failure Classes — one log line carried four unrelated defects, so its issue could never be closed; what an incident's identity is really computed from (including the second line that never reaches it) and the rule that puts the failure CLASS in the template
• The Release Gate's Denominator
• Release to Production — the whole path
• Renaming a Required Status Check
• Repository Topology
• The Release Wave — one emitter, and who resolves the digest

Contributing docs

• Start here: Authoring Documentation
• Docs Follow The Functionality — which repo a page belongs in, and what pins the rest here
• Specifying Software
• Glossary
• Developing from within MeshWeaver
• Shared Rule Blocks
• Incidental Findings — the defect you did not come for: the three tests that decide, what a filed issue owes, and why turning aside costs twice
• Governed Autonomy — do, file, govern or ask: the four responses to an instruction, and why the test is reversibility rather than importance

Licensing

• Start here: Dependency Licensing — Apache-2.0/MIT compatible only; the CI gate that enforces it
• Dependency Major Upgrades — the five things a green -warnaserror build cannot see at a major boundary (in-mesh source, authored content, the satellites that import this repo's package list, behaviour behind an unchanged signature, and rules an analyzer would have enforced had the build actually loaded it), the differential method that replaces them, and the ledger of boundaries actually crossed

Getting started

New to the platform? Read Specifying Software to learn how to write iterative specifications closely aligned with implementation, skim the Glossary for the vocabulary, then explore the full catalog of architecture topics above.