This is the absolutely horrific next stage for social media platforms:
- They're already well able to surface the most addictive short video for a specific user out of millions of real videos.
- But these millions of real videos are just darts thrown into the space of "videos that could hook the user", in the end even the best-selected of them is not perfect.
- Now, behold! AI allows to generate the perfect video to surgically hit all the switches in the viewer's brain and turn it into a zombie hooked for days on end.
Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.
Social media is not even the worst threat vector, in my opinion. It’s Anthropic/OpenAI/Google, whom people are giving an unprecedented level of access to their innermost thoughts.
We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver. And I have no doubt the likes of Dario Amodei and Sam Altman will show us, if we give them enough time.
* I suppose the GPT-4o sycophancy/AI psychosis crisis was a preview, but this was just blunt “engagement” tuning.
Carfentanyl potentcy is roughly 10,000 times morphine. The pursuit of optimized technology usually far outpaces regulation or any kind of control. As technologies are controlled, even the lessened availability of more potent types causes an enormous overall issue. Humans are obsessive-compulsive optimizers, for good and for ill.
Don't forget it's not "the platforms" doing this, it's people like you and me, maybe even people reading this message. We, they, are building this, and don't care about anything as long the paycheck comes. This is the reality.
This is very similar to last week with that mind reading startup thing. Please read the paper before commenting.
This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”.
The essence runs on having someone sit in a scanner for a couple hours watching all sorts of things, and then feeding that to a model that will then build its own representation of said data and try different things on it until it’s found what makes a certain part sing in the model.
The purpose is a generalized understanding of brain function, more or less the same way we’ve been doing it all these years. Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?
>Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.
It’s a small TV screen, placed a few feet from the larger one, that plays a continuous loop of banal, real-world scenes — a guy pouring a cup of coffee, someone getting a haircut — each lasting about 20 seconds. Whenever a youngster looks away from the Moonbug show to glimpse the Distractatron, a note is jotted down.
“It’s not mega-interesting, what’s on the Distractatron,” said Maurice Wheeler, who runs the research group. “But if they aren’t fully focused, they might go, ‘Oh, what’s that?’ and kind of drift over. We can see what they’re looking at and the exact moment when they got distracted.”
the same year as Baudrillard's Simulacra and Simulation, featuring computer generated characters, brainwashing television commercials, a light pulse gun that causes absence seizures, gun battles inside a plastic surgery clinic and an AR simulation environment, a sadistic computer, a physician who doubles as an action hero, and James Coburn giving a lecture explaining the enemy's evil plans in the opposite role that he played in the (excellent) The President's Analyst.
This time I was not so dazzled and saw it for as atrocious everyone else things it is. The minions of "Digital Matrix, Inc." manage several assassinations with the light-pulse L.O.O.K.E.R. gun but when they use real firearms they outdo Vader's stormtroopers by shooting each other. (Want to see the scene where somebody from E.Y. tells them to stick to the L.O.O.K.E.R. gun) The bad guys explain the penultimate secret to the protagonist early on but the ultimate secret is revealed in the L.O.O.K.E.R. lab which doesn't feel like a lab at all but rather a rather good room in a theme park experience where you're supposed to uncover the secret. (Contrast that to the lab Doug Trumbull outfitted in the Brainstorms movie a few years later which is packed with real surplus equipment... I've been to that lab!)
I can see it now.. The Distractatrons: a new chapter of protagonists in Transformers! The modern equivalent of evil in this day and age of ADHD and low attention span!
Also: one of the V3A animations reminds me loosely of things I saw when I was a kid, at night, shortly before I slept (though my experience then was more circular).
Isn't there that one Harry Potter warning. I think it was the potion guy who said too much luck is dangerous. I guess that is somewhat of a parallel to this. Too much positive visual stimuli is dangerous or bad.
My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).
Who is to say that the rest of us are not also vulnerable to such controlling stimuli?
There are undoubtedly 'adversarial images' that will induce certain effects in particular biological minds. It seems like this is neither good or bad inherently, just another tool.
This is already happening at scale by the social media feed algorithms. We don't need generated content to accomplish this. In a sea of user created content, plenty of it is already at peak activation.
This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"
And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.
The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.
I was asked to train a neural network do detect how much pain a mouse was in- our partner company would be responsible for hurting and filming the mice. I refused and subsequently quit- this produced no paper and I don’t know if they got someone else to do it. I probably should have done something stronger.
When skip level bosses on my last job wanted to do boneheaded things in automotive design they usually had to keep asking different engineers until they got a yay.
When it is pushed from the top it is hard to stop at ground level.
In their defence, don't shoot the messenger. Just because they published it doesn't mean that others haven't already discovered it. Better to know its possible than be completely ignorant.
I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.
Advertisements actually run the world. Imagine a world without ads. The economy would grind to a halt. It is exploitation and manipulation, but that enables creation of large capital, which leads to great things. In the past it was pyramids and temples, and in modern times it is space exploration, scientific research and other things that require huge capital. Even the current advances in AI/LLMs are made possible by mass exploitation.
Without ads and exploitation of the masses, none of these would not be possible.
Please tell me how ads help science research or why should I care about space exploration or LLMs that threaten me and hundreds of thousands of other peoples jobs?
Can’t tell if this is satirical or serious. Those ancient wonders were enabled by slave labor, treating people as “capital” is the ultimate climax of capitalism.
The title reads a lot like the lab logs you'd find in a horror game.
But for the paper itself, it seems they're using genetic optimization over predefined keywords. Wonder what would happen if they did gradient descent on the latent space directly. Is brain stimulation just not a good domain for GD?
Apart from ethically bad and evil use cases of this application, can we use it to massage the parts of brain like we do it to our bones and muscles with the help of physiotherapists?
reason I am asking it could be some relief to our brains after tedious working day, especially after heavy AI usage
Very interesting. We have an organic experiment converging to maximum stimulation in short form videos (which will become the majority of training data for future video gen models) Already approaching the capabilities of a “mood organ” from blade runner. Except usually most people don’t even make the choice to change their mood anymore.
https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_...
I wonder if the end-game of this field of research will be to run these simulations at scale using neuron-on-a-chip services such as [0] Cortical Cloud.
Fuck this and whoever is driving this type of research. We’re increasingly living in a world designed by sociopaths. What do these people think this research will be used for?
I wish we had the Hippocratic oath for STEM, or at least that they would take ethics seriously rather than an afterthought against the god of Progress at all costs.
For one, it's bound to teach us something about our brains. And while I'm not usually an optimist, I actually find it more likely that we'll take active steps to stop the evil applications of this if research like this is out in the open.
I am not sure if we should want to understand how our brain works. This is why I don't like libertarian free will people. We need to accept that we don't have free will. Otherwise we will study the brain thinking that we are above it. No. We are soon going to find out exactly how the brain works and I am not sure people understand what that means.
Will be interesting to see how strong the controlling forces can be - enough to make you miss things in direct perception like in the book, or only softer effects further up the cognition layer stack
My current theory is that these are similar to cigarettes. Nobody likes the first draft, it burns your lungs, your entire body wants to reject it. But the nicotine stimulates just the right receptors so that if you keep at it for just long enough, you'll be hooked and start disregarding the terrible taste, smell, tar in your lungs, and yellowing of your teeth.
All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.
You might still not "like" it, but the habit is what matters.
As a teenager, I tried to get addicted to cigarettes so I could stand by the cool kids and smoke with them. I started smoking 4-6 cigarettes a day but hated it so much, I couldn't continue after a week or two...
I hope my brain is also different. I also have never spent hours scrolling through short-form videos on Instagram, TikTok, Facebok, etc. I never ever walk outside with my phone in my hand, instead enjoying the view.
I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.
If you read the techbique it reads like something far less remarkable being PR’d to sound like a big deal.
The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.
The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.
"Prime Intellect, I would like you to begin stimulating the neurons of the pleasure center of my brain, one at a time, and remember the ones I report to you as being favorable."
This is the absolutely horrific next stage for social media platforms:
- They're already well able to surface the most addictive short video for a specific user out of millions of real videos.
- But these millions of real videos are just darts thrown into the space of "videos that could hook the user", in the end even the best-selected of them is not perfect.
- Now, behold! AI allows to generate the perfect video to surgically hit all the switches in the viewer's brain and turn it into a zombie hooked for days on end.
Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.
Social media is not even the worst threat vector, in my opinion. It’s Anthropic/OpenAI/Google, whom people are giving an unprecedented level of access to their innermost thoughts.
We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver. And I have no doubt the likes of Dario Amodei and Sam Altman will show us, if we give them enough time.
* I suppose the GPT-4o sycophancy/AI psychosis crisis was a preview, but this was just blunt “engagement” tuning.
Carfentanyl potentcy is roughly 10,000 times morphine. The pursuit of optimized technology usually far outpaces regulation or any kind of control. As technologies are controlled, even the lessened availability of more potent types causes an enormous overall issue. Humans are obsessive-compulsive optimizers, for good and for ill.
We are doing to our brains and pur children the same thing we are doing to pigs in cages, squezeeing every last cent until the planet burns down.
I am surprised that no comment has yet mentioned Infinite Jest (which I'm still yet to complete - it's been years since I started).
Don't forget it's not "the platforms" doing this, it's people like you and me, maybe even people reading this message. We, they, are building this, and don't care about anything as long the paycheck comes. This is the reality.
This is very similar to last week with that mind reading startup thing. Please read the paper before commenting.
This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”.
The essence runs on having someone sit in a scanner for a couple hours watching all sorts of things, and then feeding that to a model that will then build its own representation of said data and try different things on it until it’s found what makes a certain part sing in the model.
The purpose is a generalized understanding of brain function, more or less the same way we’ve been doing it all these years. Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?
>Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.
https://www.nytimes.com/2022/05/05/arts/television/cocomelon...https://www.newyorker.com/magazine/2024/06/17/cocomelon-chil...
What a world.
I know a technology like that was used ~20 years ago for ADHD. EEG feedback, as soon as the kid looks away or zones out, the movie stops playing.
I recently rewatched the movie Looker which was vastly ahead of its time in 1981
https://en.wikipedia.org/wiki/Looker
the same year as Baudrillard's Simulacra and Simulation, featuring computer generated characters, brainwashing television commercials, a light pulse gun that causes absence seizures, gun battles inside a plastic surgery clinic and an AR simulation environment, a sadistic computer, a physician who doubles as an action hero, and James Coburn giving a lecture explaining the enemy's evil plans in the opposite role that he played in the (excellent) The President's Analyst.
This time I was not so dazzled and saw it for as atrocious everyone else things it is. The minions of "Digital Matrix, Inc." manage several assassinations with the light-pulse L.O.O.K.E.R. gun but when they use real firearms they outdo Vader's stormtroopers by shooting each other. (Want to see the scene where somebody from E.Y. tells them to stick to the L.O.O.K.E.R. gun) The bad guys explain the penultimate secret to the protagonist early on but the ultimate secret is revealed in the L.O.O.K.E.R. lab which doesn't feel like a lab at all but rather a rather good room in a theme park experience where you're supposed to uncover the secret. (Contrast that to the lab Doug Trumbull outfitted in the Brainstorms movie a few years later which is packed with real surplus equipment... I've been to that lab!)
https://www.youtube.com/watch?v=cBlWxXqH8vA
> ...the Distractatron
I can see it now.. The Distractatrons: a new chapter of protagonists in Transformers! The modern equivalent of evil in this day and age of ADHD and low attention span!
As others on Telegram have said: automated search for visual superstimuli likely leads to bad outcomes.
https://en.wikipedia.org/wiki/Supernormal_stimulus
https://en.wikipedia.org/wiki/BLIT_(short_story)
Also: one of the V3A animations reminds me loosely of things I saw when I was a kid, at night, shortly before I slept (though my experience then was more circular).
Breaking news: TikTok is bad for you
On telegram?
Isn't there that one Harry Potter warning. I think it was the potion guy who said too much luck is dangerous. I guess that is somewhat of a parallel to this. Too much positive visual stimuli is dangerous or bad.
Others on Telegram? Some sort of a HN channel?
Or as SCP calls them, cognitohazards.
Also relevant: <https://www.quantamagazine.org/a-math-theory-for-why-people-...>
My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).
Who is to say that the rest of us are not also vulnerable to such controlling stimuli?
Bright lights in particular, I'm thinking: yes, normal people do find sunbathing relaxing.
There are undoubtedly 'adversarial images' that will induce certain effects in particular biological minds. It seems like this is neither good or bad inherently, just another tool.
So are opioids.
Or how a very useful tool can become a public health catastrophe.
This is already happening at scale by the social media feed algorithms. We don't need generated content to accomplish this. In a sea of user created content, plenty of it is already at peak activation.
The plan is to get content producers out of the loop to reduce revenue share and boost profits.
Optimizing for engagement is a categorically different goal than optimizing for lighting up a specific part of someone's brain.
So, AI images to maximally drive reward centers in the brain?
What could possibly go wrong?
We are really getting to the point where the tech industry must be stopped if humanity is to continue at all, let alone thrive.
This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"
> This does indeed seem comically evil.
And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.
The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.
I was asked to train a neural network do detect how much pain a mouse was in- our partner company would be responsible for hurting and filming the mice. I refused and subsequently quit- this produced no paper and I don’t know if they got someone else to do it. I probably should have done something stronger.
When skip level bosses on my last job wanted to do boneheaded things in automotive design they usually had to keep asking different engineers until they got a yay.
When it is pushed from the top it is hard to stop at ground level.
In their defence, don't shoot the messenger. Just because they published it doesn't mean that others haven't already discovered it. Better to know its possible than be completely ignorant.
Then maybe publish the results, but don't publish the "how to".
Would it be better if this was done on monkeys? Because people did that before this in silico digital brain stuff.
These aren't scientists. They are techbros. That's why it comes out like this.
I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.
I share your concern but the generalization is improper (that as solution would be infinitely far worst than the problem)
I get that people see this and think: ads and social media. My first thought was cognitive neurorehabilitation and brain stimulation.
Realistically, probably ads, but maybe not only that?
(AI start-up idea: one of our ads a day keeps dementia away! /s)
You can't say things like this on this website. On here, every new tech thing is a "progress" /s
Progress, but towards what?
Ads efficacy, of course.
Advertisements actually run the world. Imagine a world without ads. The economy would grind to a halt. It is exploitation and manipulation, but that enables creation of large capital, which leads to great things. In the past it was pyramids and temples, and in modern times it is space exploration, scientific research and other things that require huge capital. Even the current advances in AI/LLMs are made possible by mass exploitation.
Without ads and exploitation of the masses, none of these would not be possible.
Please tell me how ads help science research or why should I care about space exploration or LLMs that threaten me and hundreds of thousands of other peoples jobs?
Can’t tell if this is satirical or serious. Those ancient wonders were enabled by slave labor, treating people as “capital” is the ultimate climax of capitalism.
Not sure what you find funny about it.
>Those ancient wonders were enabled by slave labor
That is exactly the point.
Think of the shareholders and Capital. Money matters more than human, commie. /s
The title reads a lot like the lab logs you'd find in a horror game.
But for the paper itself, it seems they're using genetic optimization over predefined keywords. Wonder what would happen if they did gradient descent on the latent space directly. Is brain stimulation just not a good domain for GD?
> Is brain stimulation just not a good domain for GD?
The largest LLMs right now are at best 1% the number of parameters of a human brain.
"At best" if synapses are one parameter each, real ones are probably more than that, but nobody's entirely sure yet.
WOW! Cant wait to tell to future generations that we had voluntarily made these algorithms to manipulate and influence our own brains
Reminds me of the parrot https://www.sfsfss.com/stories2/BLIT.htm
Apart from ethically bad and evil use cases of this application, can we use it to massage the parts of brain like we do it to our bones and muscles with the help of physiotherapists?
reason I am asking it could be some relief to our brains after tedious working day, especially after heavy AI usage
Its also an interesting way to discover what that part of the brain is for, right.
what the brain needs is Default Mode Network, not more stimuli
Very interesting. We have an organic experiment converging to maximum stimulation in short form videos (which will become the majority of training data for future video gen models) Already approaching the capabilities of a “mood organ” from blade runner. Except usually most people don’t even make the choice to change their mood anymore. https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_...
What are these videos supposed to do? I watched few of them and it does nothing for me?
if it is targetting visual regions of brain and I have aphantasia (I cannot visualize anything in my mind) is that connected?
> I watched few of them and it does nothing for me?
It's so cool that you have an MRI handy to check such things.
I wonder if the end-game of this field of research will be to run these simulations at scale using neuron-on-a-chip services such as [0] Cortical Cloud.
I don't think it's a matter of if but when. Grim.
[0] https://corticallabs.com/cloud
I wonder what Meta could do with a similar technology…
But here we can start also the usual discussion about technology research for the sake of it vs calibration of possible side effects of new research
Personally i think we haven’t solve this problem and thus it’s just a matter of time until we’ll get in a non-going-back point
Wait, this is with a digital twin only? Not fMRI or webcam based?
The whole site looks AI generated. Those 3D brains are... smooth.
Inventing Snow Crash. Neat.
See Zuck, the metaverse failed, but who said the mind control part couldn't work?
Fuck this and whoever is driving this type of research. We’re increasingly living in a world designed by sociopaths. What do these people think this research will be used for?
I wish we had the Hippocratic oath for STEM, or at least that they would take ethics seriously rather than an afterthought against the god of Progress at all costs.
How can one work on that and not consider that they're pure evil?
For one, it's bound to teach us something about our brains. And while I'm not usually an optimist, I actually find it more likely that we'll take active steps to stop the evil applications of this if research like this is out in the open.
Think of the shareholders (it's time to start using physical force to stop people)
I am not sure if we should want to understand how our brain works. This is why I don't like libertarian free will people. We need to accept that we don't have free will. Otherwise we will study the brain thinking that we are above it. No. We are soon going to find out exactly how the brain works and I am not sure people understand what that means.
What in the zuck is this?
Future looks bright....
I can't wait until I see AI-generated gambling ads that are specifically created to stimulate my brain the most
Straight out of Echopraxia.
Will be interesting to see how strong the controlling forces can be - enough to make you miss things in direct perception like in the book, or only softer effects further up the cognition layer stack
That's fascinating. I wish the demo videos were longer.
Am I the only one who is avoiding even clicking the link just in case?
It is like screen saver moving patterns or corridors with a strange field of view zooming effect.
Nothing special compared to purpose made screen savers.
That’s what someone would say if they were brainwashed by them!!!
Half joking, half paranoid.
No, you are not alone.
My brain never liked vertical video, shortform content and AI slop.
Is my brain different or am I just a grumpy millenial hipster?
My current theory is that these are similar to cigarettes. Nobody likes the first draft, it burns your lungs, your entire body wants to reject it. But the nicotine stimulates just the right receptors so that if you keep at it for just long enough, you'll be hooked and start disregarding the terrible taste, smell, tar in your lungs, and yellowing of your teeth.
All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.
You might still not "like" it, but the habit is what matters.
I have the opposite theory. I burned my self out on cheap and bad image gen meme sites like 15 years ago until the point I hated memes.
Prior exposure to worse feeds gives like an analytical look on the vids rather than emotional. I am fast scanning for the joke. Or something.
As a teenager, I tried to get addicted to cigarettes so I could stand by the cool kids and smoke with them. I started smoking 4-6 cigarettes a day but hated it so much, I couldn't continue after a week or two...
You're not alone!
I hope my brain is also different. I also have never spent hours scrolling through short-form videos on Instagram, TikTok, Facebok, etc. I never ever walk outside with my phone in my hand, instead enjoying the view.
I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.
these videos were disappointing and underwhelming
If you read the techbique it reads like something far less remarkable being PR’d to sound like a big deal.
The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.
The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.
"Prime Intellect, I would like you to begin stimulating the neurons of the pleasure center of my brain, one at a time, and remember the ones I report to you as being favorable."