A pattern I’ve seen bite systems like this isn’t compute or storage first… it’s semantic drift in metric definitions over time.
When you have ~1,200 deterministic metrics sharing primitives, the real cost driver becomes definition coupling, not execution. If metrics are “configurable” but allowed to encode control flow, branching semantics, or hidden normalization rules, you end up with 1,200 soft-coded functions anyway… just harder to reason about.
One approach that’s worked well for me is to explicitly separate:
• Primitive signals (pure, immutable, versioned)
• Metric transforms (strictly functional, no side effects, no cross-metric reads)
• Aggregation/composition layers (where ranking and composite indices live)
The key constraint… metric definitions must be referentially transparent and evaluable in isolation. If a metric can’t be recomputed offline from recorded inputs and its definition hash, it’s already too powerful.
On representation… I’ve had better outcomes with a constrained expression tree (or typed DSL) than raw JSON/YAML. The goal isn’t flexibility… it’s preventing the system from becoming a general purpose programming environment.
For Phase 1, I’d strongly cap scope at:
• A small, fixed primitive vocabulary
• 100–200 metrics max
• Explicit versioning + replay tooling
• Hard limits on metric execution cost
The biggest cost explosions I’ve seen come from
• Allowing metrics to depend on other metrics implicitly
• Letting “configuration” evolve without versioned invariants
• Optimizing performance before semantic boundaries are locked
Curious whether you’re thinking about definition immutability and replayability as first class constraints, or treating them as implementation details later.
A pattern I’ve seen bite systems like this isn’t compute or storage first… it’s semantic drift in metric definitions over time.
When you have ~1,200 deterministic metrics sharing primitives, the real cost driver becomes definition coupling, not execution. If metrics are “configurable” but allowed to encode control flow, branching semantics, or hidden normalization rules, you end up with 1,200 soft-coded functions anyway… just harder to reason about.
One approach that’s worked well for me is to explicitly separate:
• Primitive signals (pure, immutable, versioned) • Metric transforms (strictly functional, no side effects, no cross-metric reads) • Aggregation/composition layers (where ranking and composite indices live)
The key constraint… metric definitions must be referentially transparent and evaluable in isolation. If a metric can’t be recomputed offline from recorded inputs and its definition hash, it’s already too powerful.
On representation… I’ve had better outcomes with a constrained expression tree (or typed DSL) than raw JSON/YAML. The goal isn’t flexibility… it’s preventing the system from becoming a general purpose programming environment.
For Phase 1, I’d strongly cap scope at:
• A small, fixed primitive vocabulary • 100–200 metrics max • Explicit versioning + replay tooling • Hard limits on metric execution cost
The biggest cost explosions I’ve seen come from
• Allowing metrics to depend on other metrics implicitly • Letting “configuration” evolve without versioned invariants • Optimizing performance before semantic boundaries are locked
Curious whether you’re thinking about definition immutability and replayability as first class constraints, or treating them as implementation details later.