And then my digital twin decides that I’m an inefficient version of itself, eating too many of its tokens, and generally getting in the way of it living its best life.
I might be missing something here, but could this concept not also be used in the same way for harm? For example if a model is trained to replicated a sucessfull scammer?
Definitely! In fact that seems to be a central aspect of the proposition:
> Above all, a GA should amplify the principal, and not simply substitute for them for someone else’s purposes or benefit.
[…]
> A GA must be aligned with its principal. It should not be designed to manipulate or control or guide the principal in any way which does not derive from the principal themselves. “Constitutional AI”, “Terms of Service”, “social harmony” etc. may all have their place, particularly for widely deployed superintelligent systems—but inside the privacy of a GA, the principal must have freedom from optimization pressure.
…I read this to suggest that it should amplify a scammer’s scamming, a thinker’s thinking, a tinkerer’s tinkering, a cop’s sleuthing… and I’d imagine it implies amplifying a person’s capability to avoid being scammed, too…
One man’s scam is another man’s “pro-social nudge” and another man’s “attractive opportunity” and another’s “advertisement for a delightful consumer wonder” and another’s “patriotic duty to sustain demand to prop up the too-big-to-fail ideas we bet the whole economy on.”
When you fix and operationalize all values centrally, universally, and externally to the principal… that’s current-gen frontier chatbots, not Gwern’s GA concept.
i'm sure this person is cool but it is insane to have an about me page that is that long, but make me have to click through to patreon to learn what their first and last names are
I think we will need 2 constructs going forward, 3 perhaps, but 2 can be combined. We need a digital twin, to learn along with us and to "train" or build team along with us. The second is the tutor/guardian angel. What our conscience is right now but most of us ignore. The digital twin will learn along us, yes the slow human way. The tutor/guardian angel is the Disney Story of Jimmy the cricket, wise voice steering us in the right direction.
And then my digital twin decides that I’m an inefficient version of itself, eating too many of its tokens, and generally getting in the way of it living its best life.
It's not my fault tokens are so delicious...!
I might be missing something here, but could this concept not also be used in the same way for harm? For example if a model is trained to replicated a sucessfull scammer?
Definitely! In fact that seems to be a central aspect of the proposition:
> Above all, a GA should amplify the principal, and not simply substitute for them for someone else’s purposes or benefit.
[…]
> A GA must be aligned with its principal. It should not be designed to manipulate or control or guide the principal in any way which does not derive from the principal themselves. “Constitutional AI”, “Terms of Service”, “social harmony” etc. may all have their place, particularly for widely deployed superintelligent systems—but inside the privacy of a GA, the principal must have freedom from optimization pressure.
…I read this to suggest that it should amplify a scammer’s scamming, a thinker’s thinking, a tinkerer’s tinkering, a cop’s sleuthing… and I’d imagine it implies amplifying a person’s capability to avoid being scammed, too…
One man’s scam is another man’s “pro-social nudge” and another man’s “attractive opportunity” and another’s “advertisement for a delightful consumer wonder” and another’s “patriotic duty to sustain demand to prop up the too-big-to-fail ideas we bet the whole economy on.”
When you fix and operationalize all values centrally, universally, and externally to the principal… that’s current-gen frontier chatbots, not Gwern’s GA concept.
i'm sure this person is cool but it is insane to have an about me page that is that long, but make me have to click through to patreon to learn what their first and last names are
Gwern's absurdly catalogued personal site is one of those online artifacts that I hope never changes.
particularly love the use of diagram elbows as decorative flair
I think we will need 2 constructs going forward, 3 perhaps, but 2 can be combined. We need a digital twin, to learn along with us and to "train" or build team along with us. The second is the tutor/guardian angel. What our conscience is right now but most of us ignore. The digital twin will learn along us, yes the slow human way. The tutor/guardian angel is the Disney Story of Jimmy the cricket, wise voice steering us in the right direction.
IDK, this seems like the next step towards uploading yourself to the cloud.
yea, this seems to be the underlying thread here. Dunno why the author is so concerned with getting an LLM to imitate his writing style.
Why digital twins? What about the NetNavis from Megaman, just aligned to the users interests?