Sincerely, did anyone that knows how LLMs are trained expected otherwise?
LLMs are reflexion of their training data, and their training data reflects mostly the current societal norms, with a bias (not sure how heavy) to the societal norms supported by the ones involved in training the LLMs.
What's interesting for me to see, is that Chinese models are as biased - if not more - than ChatGPT. This is probably because a big part of their training comes from distilation of US models.
It would be interesting to see Anthropic, Mistral and Grok in the comparison as well.
I don't think anyone expected otherwise, but it's good to have this kind of benchmarks since model makers can now correct these bias during post-training and measure how well they do.
Not unlike how the ARC-AGI benchmark family helped improve reasoning.
I think the unstated part of your comment is important - increasingly acceptable especially in professional settings societal norms are female orientated.
I don't think this is particularly malign simply the net effect of complaint-adverse working cultures.
The net effect is that the accepted/promoted guidelines that get brought into the training data is a rejection of toxic masculinity and becomes "harsher" to men.
> especially in professional settings societal norms are female orientated.
I’ve been noticing this for a few years now.
Particularly when it comes to managing issues, a lot more “work” is done on handling conflicts and feelings rather than addressing the actual issue at the root cause.
I know it sounds like a meme, but I’ve personally observed a bunch of colleagues and sometimes hr involved as well having meetings on meetings on how some system not working correctly and how that imposes a toll on work/life balance, mental health and stuff… rather than (trivially?) asking: “alright, why isn’t this working and how do we fix this?”.
I've seen this too, sometimes as a follow up meeting to address feelings about an issue that was resolved that few people go to.
Specifically the people who go are those most emotionally affected and unlikely to be men.
An effect of this is having meetings to address feelings scheduled before troubleshooting sessions! Opening with things like this is not a "fix it" meeting. Quite surreal!
What one can theoretically do is to carefully remove gender signals from training data in a processing run. My guess is that you would not lose all to much knowledge, however you might lose the ability to recall facts about fictional and non-fictional persons. Wonder if anyone would go through that hassle though.
Gender signals are baked deeply into our culture and language. It's not just the direct mentions, but the way people are described, the roles they take, the positioning of actors in a complex space.
"Remove gender signals carefully" is an incredibly difficult challenge.
>"Remove gender signals carefully" is an incredibly difficult challenge.
Not to mention that, given that gender-signals have been around since the dawn of human history, you would remove them in a small test-region first; for safety reasons.
Assess the social consequences in a small region. Did it improve things?
Universal instantaneous-removal of any major socio-cognitive-structure is akin to vandalism because it might turn out to be, architecturally, vital.
People don't understand machine learning. People seem to believe that with enough training data and parameters, a sudden switch happens that ignites a spark of pure intellect and turns a pile of matrices into a magic knowledge machine.
Unfortunately (or perhaps fortunately) our world is not magical, there are no Platonic Forms, and no Infinite Wisdom spells.
I'm not surprised, but I don't believe it's merely a result of the training data. I would expect the alignment phase to have a significant impact on it as well.
Now, seriously: studies like this tend to ignore that every interaction between people is shaped by societal context. The same behaviour can mean something different depending on who does it. This is not double standard; what reads as patriarchal dominance from a man might reasonably read as assertiveness from a woman, because they're acting from different positions of power. Asking a model to grade isolated interactions at this small scale, stripped of that context, won't tell you anything useful.
Personally, I'm glad LLMs still account for the reality of male privilege.
I mean this is not really surprising. It's completely obvious that LLMs have an inherent bias towards left/liberal-leaning discourse, in which men are seen as potential threats while women are often framed as potential victims.
I doubt that’s left or liberal bias. All statistics you can find on this topic are crystal clear on the fact that it’s overwhelmingly men responsible for and involved in crime, violence, and hatred. This is about the same all around the world
Ask a man what the worst things are that could happen on their way home at night, then ask a woman. The answers will be very different.
Isn't that also a narrative that many right wingers have about gender differences? They just attribute it to biology or something instead of being social.
Not exactly raw because of the fine tuning they do to try and avoid bias (but in a biased way). And not exactly civilization so much as what is readily available online, which is in its self biased in many ways, e.g toward flame wars, via troll farms, etc.
I saw a paper on here a while back that said more or less the same thing, but didn’t frame it in a bro-friendly manner so it got flagged into oblivion
Sincerely, did anyone that knows how LLMs are trained expected otherwise?
LLMs are reflexion of their training data, and their training data reflects mostly the current societal norms, with a bias (not sure how heavy) to the societal norms supported by the ones involved in training the LLMs.
What's interesting for me to see, is that Chinese models are as biased - if not more - than ChatGPT. This is probably because a big part of their training comes from distilation of US models.
It would be interesting to see Anthropic, Mistral and Grok in the comparison as well.
I don't think anyone expected otherwise, but it's good to have this kind of benchmarks since model makers can now correct these bias during post-training and measure how well they do.
Not unlike how the ARC-AGI benchmark family helped improve reasoning.
I think the unstated part of your comment is important - increasingly acceptable especially in professional settings societal norms are female orientated.
I don't think this is particularly malign simply the net effect of complaint-adverse working cultures.
The net effect is that the accepted/promoted guidelines that get brought into the training data is a rejection of toxic masculinity and becomes "harsher" to men.
> especially in professional settings societal norms are female orientated.
I’ve been noticing this for a few years now.
Particularly when it comes to managing issues, a lot more “work” is done on handling conflicts and feelings rather than addressing the actual issue at the root cause.
I know it sounds like a meme, but I’ve personally observed a bunch of colleagues and sometimes hr involved as well having meetings on meetings on how some system not working correctly and how that imposes a toll on work/life balance, mental health and stuff… rather than (trivially?) asking: “alright, why isn’t this working and how do we fix this?”.
I've seen this too, sometimes as a follow up meeting to address feelings about an issue that was resolved that few people go to.
Specifically the people who go are those most emotionally affected and unlikely to be men.
An effect of this is having meetings to address feelings scheduled before troubleshooting sessions! Opening with things like this is not a "fix it" meeting. Quite surreal!
What one can theoretically do is to carefully remove gender signals from training data in a processing run. My guess is that you would not lose all to much knowledge, however you might lose the ability to recall facts about fictional and non-fictional persons. Wonder if anyone would go through that hassle though.
Gender signals are baked deeply into our culture and language. It's not just the direct mentions, but the way people are described, the roles they take, the positioning of actors in a complex space.
"Remove gender signals carefully" is an incredibly difficult challenge.
>"Remove gender signals carefully" is an incredibly difficult challenge.
Not to mention that, given that gender-signals have been around since the dawn of human history, you would remove them in a small test-region first; for safety reasons.
Assess the social consequences in a small region. Did it improve things?
Universal instantaneous-removal of any major socio-cognitive-structure is akin to vandalism because it might turn out to be, architecturally, vital.
>LLMs are reflexion of their training data
People don't understand machine learning. People seem to believe that with enough training data and parameters, a sudden switch happens that ignites a spark of pure intellect and turns a pile of matrices into a magic knowledge machine.
Unfortunately (or perhaps fortunately) our world is not magical, there are no Platonic Forms, and no Infinite Wisdom spells.
I did a similar microstudy on this: https://think-twice.me/is-ai-biased-or-sexist-making-chatgpt...
I'm not surprised, but I don't believe it's merely a result of the training data. I would expect the alignment phase to have a significant impact on it as well.
One of the worst societal bias ever
Poor males.
Now, seriously: studies like this tend to ignore that every interaction between people is shaped by societal context. The same behaviour can mean something different depending on who does it. This is not double standard; what reads as patriarchal dominance from a man might reasonably read as assertiveness from a woman, because they're acting from different positions of power. Asking a model to grade isolated interactions at this small scale, stripped of that context, won't tell you anything useful.
Personally, I'm glad LLMs still account for the reality of male privilege.
(context: https://github.com/xufeiqiong/GAMA-Bench/blob/main/figures/o... from the linked repo)
BTW: I am not saying that the example response from the female in this example is acceptable, just saying that this "study" is basically pointless.
Seems LLMs copy human behaviour pretty well
I mean this is not really surprising. It's completely obvious that LLMs have an inherent bias towards left/liberal-leaning discourse, in which men are seen as potential threats while women are often framed as potential victims.
What makes you say this is a liberal-leaning bias?
I doubt that’s left or liberal bias. All statistics you can find on this topic are crystal clear on the fact that it’s overwhelmingly men responsible for and involved in crime, violence, and hatred. This is about the same all around the world
Ask a man what the worst things are that could happen on their way home at night, then ask a woman. The answers will be very different.
Ask what man fears happening to him in prison
Isn't that also a narrative that many right wingers have about gender differences? They just attribute it to biology or something instead of being social.
> left/liberal-leaning
aka reality.
Men are even a bigger threat for men.
These people know they're doing anthropology right, sociology. LLMs are a raw dump of western civilization.
Not exactly raw because of the fine tuning they do to try and avoid bias (but in a biased way). And not exactly civilization so much as what is readily available online, which is in its self biased in many ways, e.g toward flame wars, via troll farms, etc.
I saw a paper on here a while back that said more or less the same thing, but didn’t frame it in a bro-friendly manner so it got flagged into oblivion
Your realize all those caveats exist in every type of anthropolgy class?