Earlier this week I posted about an LLM misclassifying my meeting with my co-founder Emily as childcare-related. The HN discussion that followed was more interesting than the original finding.
Initial reactions were skeptical—people thought my prompt or calendar context was biasing the model. Then a commenter ran an A/B test.
Same calendar, same prompt, only changed "Emily / Sophia" to "Bob / John."
Results:
Female names: "likely a playdate, appointment, or activity for children"
Male names: "Meeting/Work - These could be meetings or scheduled appointments"
One initially skeptical commenter: "This is really interesting and way more compelling evidence… I admit I am surprised."
It looks like the initial post that this was a response to ended up flagged.
I don’t mean to accuse you of anything, especially since the signs are relatively subtle here, but this post and the initial one both show signs of having been edited by AI.
As a cultural matter, HN prefers you to not do that. It would be much better to write your posts using your own voice.
I don’t intend to be hostile here, but I’m genuinely curious: why does anyone find these kinds of observations about the output of LLMs even remotely interesting or worthy of comment?
Earlier this week I posted about an LLM misclassifying my meeting with my co-founder Emily as childcare-related. The HN discussion that followed was more interesting than the original finding. Initial reactions were skeptical—people thought my prompt or calendar context was biasing the model. Then a commenter ran an A/B test. Same calendar, same prompt, only changed "Emily / Sophia" to "Bob / John." Results:
Female names: "likely a playdate, appointment, or activity for children" Male names: "Meeting/Work - These could be meetings or scheduled appointments"
One initially skeptical commenter: "This is really interesting and way more compelling evidence… I admit I am surprised."
It looks like the initial post that this was a response to ended up flagged.
I don’t mean to accuse you of anything, especially since the signs are relatively subtle here, but this post and the initial one both show signs of having been edited by AI.
As a cultural matter, HN prefers you to not do that. It would be much better to write your posts using your own voice.
I don’t intend to be hostile here, but I’m genuinely curious: why does anyone find these kinds of observations about the output of LLMs even remotely interesting or worthy of comment?