I worry deeply because Meta connects a vast amount of humanity. The damage they've done already is immense; the damage they could do to civilisation (democracy, our trajectory towards freedom and peace, the post-WW2 vision of the West) is even worse. I've read [0] Zuckerberg wants to think of himself like one of the early Caesars (thus explaining the haircut) with a focus on absolute domination; what would such a man with such a fixation be willing to do?
that was the first stab at it; but it's pretty clear that you try to get good data, especially when you see how much the open source chinese models are doing with far less compute and parameter sizes. These are the open source datasets: https://huggingface.co/datasets/HuggingFaceFW/fineweb
I don't think they do that anymore, but at the beginning, yes, they threw all kind of bullshit in there to get a proper gradient.
surely it depends on how the data is labeled or classified. When a model is trained on fiction and medical journals, it doesn't assume that a healing potion is the right answer for a runny nose.
As an interested outsider to U.S. politics (i.e. Canadian), this seems like a political choice that may indicate tech billionaires are finally picking a side.
For the last few decades, big money has donated heavily to both U.S. political parties. It didn't matter which party won so long as they were deeply indebted to you and utterly reliant on you to fund their next campaign. This is why, for a very long time, while both U.S. parties often promised different things, what they actually did when in power was usually very similar to each other. This led to a gradual erosion of regulations and tax laws and a corresponding increase in both corporate freedom and wealth inequality.
The current administration is more transactional than any before, even when compared to Trump's first term. Power is for sale. Today, billionaires have opportunities with the Republican party that they've never had before. Meanwhile, the Democratic party may be shifting to the left, tapping into massive, pent-up anti-billionaire sentiments. It's long been the case that people like Sanders and AOC, while very vocal, had zero chance of assuming leadership roles because they scare away big money. That may be changing. There's so much pent-up desire for reform that left-leaning (albeit still very right-wing by international standards) leaders might be so immediately popular that they can afford to scare away billionaires. For a very long time the Democrats seemed almost happy to lose, because what they promised to voters was not what they were in a position to deliver without losing their financial backers. Losing gave them an excuse not to do what they promised. If the left-wing elements take over for an election cycle and actually win without being beholden to big money, that might lead to big changes in what money is allowed into election campaigns and drastically curtail the amount of influence that can be bought. This prospect, obviously, scares those are are accustomed to buying power.
Meta and other AI companies are strip-mining everything they can whether they have permission to do so or not. They also tend to be incredibly secretive about everything they do. Massive data centres are popping up in people's backyards without them hearing of it until shovels are in the ground. Everything is being done under NDA's. This deal is public, so it should be viewed as a deliberate statement. Meta intends to throw extra weight behind Newsmax's content because their leadership approves of it. What this says to me is that Zuckerberg has chosen to side with MAGA because it benefits him, while he does not like the direction the Democrats are headed in.
I worry deeply because Meta connects a vast amount of humanity. The damage they've done already is immense; the damage they could do to civilisation (democracy, our trajectory towards freedom and peace, the post-WW2 vision of the West) is even worse. I've read [0] Zuckerberg wants to think of himself like one of the early Caesars (thus explaining the haircut) with a focus on absolute domination; what would such a man with such a fixation be willing to do?
[0] https://romanempiretimes.com/is-mark-zuckerberg-just-a-roman...
> the damage they could do to civilisation..
Not could, already have.
That ship sailed, reached it's destination, had a leisurely visit to the chandlery and is on the way back to do it again.
I was under the impression that all of them try to train their models on all data they can possible ibtain.
I'd think what ever data this Newsmax has - it would be in all datasets by now or soon enough. Was I wrong?
that was the first stab at it; but it's pretty clear that you try to get good data, especially when you see how much the open source chinese models are doing with far less compute and parameter sizes. These are the open source datasets: https://huggingface.co/datasets/HuggingFaceFW/fineweb
I don't think they do that anymore, but at the beginning, yes, they threw all kind of bullshit in there to get a proper gradient.
The thing that bothers me about this article is that there was no attempt to explain why Meta would want to train on this content.
The reason should be obvious. Powerful people would complain about unfairness/bias otherwise.
surely it depends on how the data is labeled or classified. When a model is trained on fiction and medical journals, it doesn't assume that a healing potion is the right answer for a runny nose.
> it doesn't assume that a healing potion is the right answer for a runny nose.
Sure, that would be absurd.
The treatment is raw milk and peptides.
Yes, I need to have a full picture from the article for this reason, to counter my initial baises.
Maybe their goal is to be able to point to the untrustworthiness of the source material as cover when they're accused of hallucination.
Philip K Dick style the invasion of the fake humans, on and on. Programmed dis-reality.
But when it's trained on far-left media outlets, where's the outrage?
As an interested outsider to U.S. politics (i.e. Canadian), this seems like a political choice that may indicate tech billionaires are finally picking a side.
For the last few decades, big money has donated heavily to both U.S. political parties. It didn't matter which party won so long as they were deeply indebted to you and utterly reliant on you to fund their next campaign. This is why, for a very long time, while both U.S. parties often promised different things, what they actually did when in power was usually very similar to each other. This led to a gradual erosion of regulations and tax laws and a corresponding increase in both corporate freedom and wealth inequality.
The current administration is more transactional than any before, even when compared to Trump's first term. Power is for sale. Today, billionaires have opportunities with the Republican party that they've never had before. Meanwhile, the Democratic party may be shifting to the left, tapping into massive, pent-up anti-billionaire sentiments. It's long been the case that people like Sanders and AOC, while very vocal, had zero chance of assuming leadership roles because they scare away big money. That may be changing. There's so much pent-up desire for reform that left-leaning (albeit still very right-wing by international standards) leaders might be so immediately popular that they can afford to scare away billionaires. For a very long time the Democrats seemed almost happy to lose, because what they promised to voters was not what they were in a position to deliver without losing their financial backers. Losing gave them an excuse not to do what they promised. If the left-wing elements take over for an election cycle and actually win without being beholden to big money, that might lead to big changes in what money is allowed into election campaigns and drastically curtail the amount of influence that can be bought. This prospect, obviously, scares those are are accustomed to buying power.
Meta and other AI companies are strip-mining everything they can whether they have permission to do so or not. They also tend to be incredibly secretive about everything they do. Massive data centres are popping up in people's backyards without them hearing of it until shovels are in the ground. Everything is being done under NDA's. This deal is public, so it should be viewed as a deliberate statement. Meta intends to throw extra weight behind Newsmax's content because their leadership approves of it. What this says to me is that Zuckerberg has chosen to side with MAGA because it benefits him, while he does not like the direction the Democrats are headed in.
[dead]