Shadow Foundations

Adapted from textbook chapters 1–3. Shadow is a living development environment and a verification system built on it. Keeping these two jobs distinct prevents the moving environment from invalidating a controlled experiment.

A moving production counterpart

A trusted materializer reads approved production sources, sanitizes them and publishes immutable validated base revisions. shadow-current names the latest accepted base. A developer workspace forks one base into isolated writable resources; a verification run pins that base until the run ends. Exploratory work may explicitly refresh or recreate from a newer base.

There are two trust tiers, dev and prod. A workspace is an execution instance in dev, not a third environment enum. It has its own resource manifest, credentials, lease and effect authority. Sharing an auth host or process is an implementation limitation to assess, not proof of equivalent isolation.

State and traffic are separate materialization paths. Recent requests may reference old rows, objects, relations and configuration. A seven-day traffic policy does not justify deleting every older row. Preserve relationship closure, cardinality skew, hot entities, schema/index/sequence state, flags and the authorization model required by the admitted workload. Start with an approved complete sanitized dataset when it is simpler than a correct subset.

What an experiment fixes

(next_state, observations, effect_intents)
    = Run(artifacts, initial_state, admitted_inputs;
          dependency_policy, nondeterminism_control)

Artifacts include actual binaries, client code, framework/runtime compatibility and configuration. Initial state includes all participating resources. Inputs include the selected boundary's requests or events. Dependency policy defines which effects execute. Nondeterminism control defines what clocks, randomness, scheduling and completion order are recorded or simulated.

For a differential experiment, fork baseline and candidate from equivalent pre-state B0, admit the same input and keep the materializer's advance to B1 outside both executions. Applying production CDC into a running experiment and replaying the operation can apply the same change twice. Likewise, executing a queue producer and injecting its recorded output executes two input paths.

Yesterday's traffic against today's sanitized base is a production-shaped test. Historical reproduction additionally requires historical pre-state or a qualified recorded boundary. A latest-state snapshot is not automatically that pre-state.

Continuous shadow comparison uses explicit windows and reset/advance boundaries. Moving state is valuable for discovery; pinned state is necessary for repeatability. A result states which one it actually used.

Three execution profiles

These profiles compose through explicit policies; they are not a collection of loosely interacting flags.

The execution cut

An execution cut chooses where effects remain real and where results are substituted. For every operation/resource/principal/run, compile one policy and one substitution owner:

HTTP method or idempotency is not an authority model: an isolated non-idempotent insert can be safe; an idempotent production write is still outside a dev workspace's authority. Recorded AI output checks orchestration, not current model quality. A proxy is an instrumentation boundary, not a sandbox for hostile code.

Multiple layers can observe one operation, but only the selected boundary substitutes the effect. Missing/ambiguous matching fails the replay rather than silently reaching a real dependency. Isolation and comparison contracts are expanded in Workspaces and Capture and replay.

Reader check

For any proposed test, name the artifact vector, pre-state, admitted input boundary, effect policies and uncontrolled nondeterminism. If those cannot be named, the result may be useful exploratory evidence, but it cannot yet be a repeatable verification claim.