2 points | by steer_dev 20 hours ago ago
1 comments
AI Slop (apologies, filler) isn't just annoying; it is In-Band Signaling Noise. The model mixes control signals (persona) with data (payload).
Prompting ("Be concise") is brittle. I switched to Shannon Entropy.
The Hypothesis: Code/Data is mathematically "messy" (High Entropy). Slop is smooth and predictable (Low Entropy).
I wrote a filter that blocks responses if entropy dips below ~3.5.
The Payoff: It captures the blocked slop as a dataset for DPO. I use the math to gather data today so I can fine-tune a natively quiet model tomorrow.
Repo: https://github.com/imtt-dev/steer
AI Slop (apologies, filler) isn't just annoying; it is In-Band Signaling Noise. The model mixes control signals (persona) with data (payload).
Prompting ("Be concise") is brittle. I switched to Shannon Entropy.
The Hypothesis: Code/Data is mathematically "messy" (High Entropy). Slop is smooth and predictable (Low Entropy).
I wrote a filter that blocks responses if entropy dips below ~3.5.
The Payoff: It captures the blocked slop as a dataset for DPO. I use the math to gather data today so I can fine-tune a natively quiet model tomorrow.
Repo: https://github.com/imtt-dev/steer