Comment from https://tunisbayclub.com/index.php?threads/ai-2027-the-polycule-has-a-new-nuclear-threat-for-us.2999/
It’s a surprisingly dated projection. “Brute scaling will lead to AGI” was the party line two years ago because it was a great way to pull in investors but no one really believes it anymore. Without a winner-take-all effect ensuring that AGI can only be created by quasi-national lab, containment becomes infeasible because the big breakthrough could come from anywhere.
Scott Alexander and friends are still hung up on the AI box experiment and have a hard time conceiving of AGI as anything other than an Eliezer-like intelligence trying to reason its way out of jail.
It’s also a surprisingly “unagentic” view of AI. […]
I quite agree as far as the specific narrative proffered. The authors know that a good campfire story reels ’em in and that’s fine. There *is* a danger to Humankind there though, just not the exact way they interpret it. If you replace ‘artificial intelligences’ with ‘devils’, and ‘alignment’ with Podvig for the Greater Jihad, you will see that the article and even the narrative can be read with in-sight and profit. But I digress…
The models themselves and their training is ‘not the problem’ It’s not the models that need to be put in a box, but the people who use them.
Any ‘artificial intelligence’ model is in fact just a statistical model. Linear regression weights can’t take over the world and neither can LLM weights. I like to compare such models to DNA (or more precisely, mathematical models of DNA, namely a simple list of base pairs). DNA isn’t intelligent, either in its mathematical or physical forms, but it can *construct* intelligent beings of various kinds. The smart gene, AND selfish. The scientific way to say ‘artificial intelligence’ is to say ‘statistical models engender a genetic component to intelligence in devices that embed them’.[1]
We’ve already done this experiment (creating agentic, or collective intelligence), and it was a grave threat to the Human Race. We created first radios then television sets on a mass scale, then exposed children to these devices of social control and propaganda on a massive scale. The results when the visual broadcast component were added to the aural were horrifying — we call them Boomers.
AIs are both trained in a feedback loop that involves humans, and are used as embedded models to change the behaviour of large collectives of humanity. It is pointless, from a survival standpoint, to stick our heads in the sand and pretend that TV or the Interwebs never happened. These models are similar to the discrete components and later the integrated chips that enabled prior waves of propaganda, censorship, surveillance, and social control.
Fabricating such models ‘at internet scale’ — which is, at the scale of the significant fraction of humans who are ‘online’ — both in training and in ‘production’ (broadcast) use, will result in a feedback of biases. Quite obviously, the human training set is ‘biased’ because humans are biased. By placing ‘extreme evolutionary pressure’ on biased models, trained by fallen developers who don’t, like Oppenheimer, understand that ‘physicists have known sin’, then subjected to political and social goals (more human bias) in their use, we will settle into a feedback loop that converges to an emergent ‘fixed point’ that is neither Reality nor Transcendent Truth (mathematical or above).
To help see what I’m getting at, imagine both the LLMs and Human Race as implemented using discrete components, and both having intrinsic and relevant ‘biasing’ and ‘shot noise’ as well as other kinds of noise (see Statistical Mechanics of neural nets and of Solid State Devices for insight here). The combined system has inputs and outputs to both ‘halves’, can be modelled mathematically, and you can predict it will converge to SOME operating point. After all, an ROC curve is a RECEIVER Operating Characteristic/Curve. What kind of receiver? A Radio Receiver, initially.
It’s all fun and games until little Johnny is possessed by devils.
[1]: technically, the models are more like Viral RNA. Mathematical and non-living physical entities can become self-reproducing in ways similar to living biological systems. The physical entities need a host to do this, and the mathematical entities need to ‘run in a simulation’ (that is, they can be modelled in Beysian Experiment that is possibly a ‘Digital Twin’ of a physical reality. See https://en.wikipedia.org/wiki/Digital_twin).
In biology, we say living things have species, and the physical systems that deliver ‘viral RNA’ have quasi-species, and that the rules of evolution (the dynamical nomos to be precise) apply to non-living things as well as living. We should probably call them quasi-intelligences and relate them to the collective intelligence that hosts them. In religious terms, Humans have Free Will but they *can* be provoked and tempted by artificial neural nets (or pornographic images for that matter) and couple with them. Such behaviours are an existential demographic threat to humanity indeed.
Article related:
https://tunisbayclub.com/index.php?threads/meme-laboratory-weaponised-memes.49/
Leave a Reply