Detecting "AI Slop" with Shannon Entropy (Python)

(steerlabs.substack.com)

2 points | by steer_dev 20 hours ago ago

1 comments

  • steer_dev 20 hours ago

    AI Slop (apologies, filler) isn't just annoying; it is In-Band Signaling Noise. The model mixes control signals (persona) with data (payload).

    Prompting ("Be concise") is brittle. I switched to Shannon Entropy.

    The Hypothesis: Code/Data is mathematically "messy" (High Entropy). Slop is smooth and predictable (Low Entropy).

    I wrote a filter that blocks responses if entropy dips below ~3.5.

    The Payoff: It captures the blocked slop as a dataset for DPO. I use the math to gather data today so I can fine-tune a natively quiet model tomorrow.

    Repo: https://github.com/imtt-dev/steer