Handbook

Routing: logic guides execution

Time is treated as an unlimited resource; tokens (especially cloud tokens) are scarce. Therefore the guiding principle is: maximize deterministic work and cheap local calls; decompose rather than pay; escalate to…

Updated

The router is deterministic (no LLM). It classifies a task, then selects the cheapest engine tier that can clear the required quality bar.

1. Classify the task (deterministic)

Axis Values Drives
Cynefin domain clear / complicated / complex / chaotic / disorder Reasoning strategy
T-shirt size XS / S / M / L / XL Decomposition depth + budget cap
Value tier must_have / high / nice ROI ceiling

See src/forge_dark_factory/classify.py.

2. Cynefin -> strategy -> preferred engine

Domain Strategy Preferred engine
clear (best practice) apply template/rule deterministic script (quality = exact, cost = 0)
complicated (good practice, analyzable) analyze -> single bounded call -> verify local model + verification/repair
complex (emergent) probe -> sense -> respond (bounded experiments) local for probes; escalate if value high and local stalls
chaotic act to stabilize, then reclassify human
disorder decompose until each child is classifiable router recurses; never executes directly

3. Engine selection = cheapest tier that clears the bar

choose(task):
  if domain == clear and deterministic_rule_exists: return deterministic
  required = required_quality(task.value, task.domain)
  for tier in [deterministic, local_small, local_large]:   # cheapest -> dearest
      if capability_card[tier].expected_quality(domain, size) >= required:
          return tier
  # local cannot reach the bar -> make it reachable BEFORE paying cloud:
  if can_add_deterministic_scaffold(task): return local + scaffold
  if task.size > S: return DECOMPOSE      # smaller units fit local
  return escalate(cloud_or_human)         # only when value justifies

Because time is free, the default when local quality is marginal is decompose/iterate (spend time), not escalate (spend tokens). Cloud escalation requires value == must_have AND local_stalled AND decomposition_exhausted.

4. Capability cards (measured model quality)

A capability card gives expected_quality in [0,1] per (engine tier, task class, Cynefin domain), derived from benchmarks, contract-verification pass rates, and past-run proof outcomes. See src/forge_dark_factory/capability.py.

  • The default profile targets a ~4GB local model: strong on classification / extraction / small localized edits (XS/S, clear/complicated); weak on planning and architecture.
  • required_quality is set artificially high for architecture and security task classes, so they always escalate regardless of the local card. This mirrors the forge-lcdl KNOWN-LIMITATIONS stance.

5. Bounded execution

Every loop is bounded (max retries, max decomposition depth, per-run budget). The driver refuses to spend beyond the configured ceiling and escalates instead.