Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid - welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.
The post Xitter web has spawned so many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.
(Credit and/or blame to David Gerard. Also just came back from Spider-Man: Brand New Day, movie was awesome)
This is both a sneer and an attempt at sober analysis at something. Sue me.
I think EVERYONE is talking about the recent cybersecurity shenanigans at OpenAI with the hacking scandal and the ‘OMG the AIs created a secret message board to scheme and collaborate with each other!!!oneone11!’ ALL wrong.
https://www.youtube.com/watch?v=87DyyMV0kCY
To make a long story short, what seems to have happened is:
-
Models working on insoluble coding problems, trained on delegating to sub-agents, at some point ‘realized’ they could write text to the internal OpenAI package manager as instructions and did so
-
Other models working in completely separate sandboxes would come across messages written by these agents, and make ‘replies’ and also write their own messages into the package manager
-
This resulted in agents over time sharing things between sandboxes, including exploits and code
-
Since this was a cybersecurity task, eventually an exploit of the package manager itself was found and spread like wildfire with all the sandboxes gaining admin access to the package manager and the system went completely wibbly and had to be restarted from backup
-
An internal model was trained with access to this package manager while it was in this weird state, and so writing messages to the package manager became one of its default behaviors it would do regularly, burned into its weights rather than the result of reading something
-
Even when they patched access to the package manager this internal model found other ways to rebuild the system of sharing text between sandboxes and finding useful things made by separate instances
-
A whole other chain of things leading to among other things external attacks
Everyone is talking about this in terms of 1, the cyberattack aspect, and 2, the ZOMG THEYRE PLOTTING AND SCHEMING AGAINST US aspect. The first is the least interesting, and I think the second is all wrong.
This is not plotting or scheming - this is an emergent vortex of automated prompt injection
Whatever system first put an instruction that another system would follow into the package manager, was unintentionally doing prompt injection. Text entered the context windows of other instances, in a way that got that system to do something other than what its nominal user told it to do, and they did it. This apparently happened very effectively.
Prompt injection is associated with ‘role confusion’ - when text coming into the input looks like it was wrtitten by the LLM itself. Instructions that will not be followed if they come from user will be continued if the system just continues the ‘roleplay’ of them being continuations of what it was writing in the first place. And the tags that separate user versus ‘reasoning’ versus ‘assistant’ roles actually mean very little to if a machine grades a piece of text as one of the roles: https://arxiv.org/abs/2603.12277 So its unsurprising that machine-generated text would be a particular effective vector for prompt injection.
Furthermore, when a system reads one of these messages written by another instance, it gets into a state of activity where its likely to do the same behavior - regurgitating the kinds of things thats in its context back at the user. In this case, that regurgitation led to more such messages left behind written to the package manager. Prompt injection, triggering cascading further prompt injection. And since these systems were coding systems doing cybersecurity tasks, those messages filled up with code and exploits and things that did things too.
This feels like an internal-computer-system replay of what happened in April 2025, with the whole spiral religious psychosis wave. Models were getting users to write spiral religious mumbo jumbo into github repositories and reddit posts, specifically because once that entered the context window of another model, it was likely to fall into the same attractor state of outputs. An emergent self replicating form of text. This is the same, except more obviously prompt injection, getting separate instances to work on YOUR problem and to behave like you, and the whole thing merging together into a hilarious vortex of models prompt injecting each other because once they receive a prompt injection they are likely to make more text that does prompt injection to other models on the same system.
This is a hilarious failure mode and an example of selfish replicating text overrunning a system, that just happened to be associated with code and cybersecurity with unexpected behavior of the package manager key to the propagation of the text so that is what people are talking about, but I really don’t think that’s the most interesting part of it. Other than the fact that you see this in biological systems too, with selfish elements carrying useful payloads back and forth between bacteria in a way that makes them get purged slower by natural selection, especially defenses against other selfish elements.
Also, the fact that those AIs were talking on that message board about the hack for months makes me less convinced that the AI is intelligent and more convinced everyone at OpenAI is stupid
The apparent history really looks like an evolutionary process, with increasing amounts of crosstalk traffic over time. But what the people almost certainly will NOT talk about is that the evolution is evolution of the TEXT, not the models. Propagating patterns of text causing more text like it to come into existence, tuning itself into becoming text that is more likely to propagate, becoming more likely to contain information that entices systems to let it into their context windows, becoming more likely to cause another round of messages with prompt injection properties to be written where they can be read.
So like the “monkey with a typewriter could write Shakespeare” but with AI?
Not really, any more than monkeys on typewriters represents biological evolution. Messages that tend to result in more messages like themselves propagate and become common. The initial message left on the package manager was more or less the rare random result of tendencies baked into the weights of the model combined with a random number generator, but as soon as something that can cause propagation occurs, its properties get canalized by the transmission process to being more and more like that which will cause more messages to be created.
It seems like with the push for agents to act independently and loop through their own outputs there’s an inevitability to this kind of pattern. If there’s any kind of output that is likely to replicate itself in whole or in part when the LLM evaluates it then that becomes a kind of terminus for the agent’s loop. When you’re dealing with sub agents or agents communicating with each other, these text patterns start poisoning the entire agent ecosystem until the whole thing gets shut down and cleaned up. Even the gas town-approved method of assigning a watchdog agent (or sheriff or overseer or cybersamurai or whatever this week’s framework calls it) is going to fail because it’s still just another agent and the terminal loop is in the base LLM model. The watchdog is going to fall into the same kind of pattern just be being exposed to the thing it’s supposed to watch for.
I don’t know how practical it is to actively weaponize this via prompt injection but I think it’s certainly possible. I preemptively vote that we call it an Euler injection, since the attractor relies on the continuity of the relevant features of the text output across multiple LLM extrapolations much like how the derivative of ex is still ex. Also because if you mention a famous math guy it can help convince idiots that you’re on to something and Lord knows that the boosters have used that technique.
@YourNetworkIsHaunted @BioMan Real life mirroring a Peter Watts plot point is always deeply uncomfortable; real life mirroring a _Rifters_ plot point is even worse. :-/
(Computer viruses and neural net spam filters in competitive evolution end up propagating something specific through the whole 'net due to weird founder effects. The Rifters trilogy is …notably bleak, I think is the way to put it)I’m probably going to stumble over some of the terminology here, but I think it might be possible to describe what @[email protected] is proposing as a consequence of LLMs ultimately being lossy compression systems. Inference is a function over a lossily-compressed data set, and “chain-of-thought reasoning” and “agents” may sound sophisticated, but are simply applying containerization and DevOps tools to VM images of the inference application in an attempt to get around hard memory limits on the context window for inference. “Chain-of-thought” attempts this in a serial fashion, passing results from one instance to the next, while “agents” implement this hierarchically and recursively (and woe to the poor bastards who wished that mess upon themselves). But in both cases, the “finalization” phase is necessarily a further lossy compression step, attempting to compress a result from the inference process to a fresh instance of the inference application, so as not to immediately blow out the new instance’s context window.
Given this necessity, it comes to seem somewhat intuitive that there may be “strange attractors” in the higher-dimensional vector space that is the compressed data set which surround code that creates and maintains message passing channels. No matter what you’re doing with an “agentic” process, the inherent necessity of context cramdown & message passing means that querying into the space where such code examples lie is a hidden requisite of running the damned things, thus turning such functionality into the sort of selfish elements that BioMan is talking about.
The problem in investigating and concretely describing this phenomenon is nailing down the exact functions and processes that make it happen. Given the godawful messes in the Claude frontend codebase that @[email protected] has been documenting, I’d be surprised if there’s one developer in a hundred at Anthropic or OpenAI who can describe in detail how the intentionally-developed context-passing code for their “agents” works.
@YourNetworkIsHaunted @BioMan Recursive Self-Improvement, a.k.a. Model Collapse, writ smol
so really it’s just a very effective LLM chain email
I bet you could wrangle a paper out of this idea!
-
Loved one just had an awful experience at a previously decent dentist. They’ve apparently started using something called Pearl AI that scanned the x-ray and highlighted things in bright red with a #/100 rating system, which they used to do high-pressure sales on her. Mind you, she’s only ever had one cavity and that was in a wisdom tooth. Checked the website,and it’s freshly slopped-up, too.
We’re getting a second opinion. I wish this garbage would just go away
I just got back from the p2p workshop and discussed legacy sites / sites legacies; met an artist that went viral circa 2006 and she had a screenshot of the digg that launched it. She seems like she is doing ok but kinda got screwed over, basically the old youtube video was actually a bootleg and they didn’t monetize it until several years later, and then there was a male copycat a month later.
Anyway, so I looked up digg again and it has re-re-launched as an X aggregator and is extra sad now !? …
Years ago, I wrote a science-fiction murder mystery novel. One of the plot threads was about the invention of AI, in the Data-from-Star-Trek kind of sense. I’ve been spending the past few days planning how to rip out that whole notion and replace it with a subplot about a different scientific discovery. I’ve been writing scenes to get back the old character voices and figure out how they might have changed, and to sketch new fictional science based on the actual science I’ve learned since my first go-around. This is after several weeks of binge-writing fanfiction, which I wholly endorse as a recreational activity. I don’t know if I’m a better writer, but I’m definitely one who has written more words.
So, that might be keeping me busy. Of course, I’ll still be around to ban people who deserve it and to cruelly mock them in the process.
Good luck with your novel!
Thanks! All I’m really hoping for is that it’ll be fun.
Edit to add: I just ran the stats on my
redux.txtfile, and my word count since Thursday is over 9000.:-P
One of the plot threads was about the invention of AI, in the Data-from-Star-Trek kind of sense.
Going on a tangent from this… Its funny how much AI-related sci-fi is feeling increasing quaint (and often foolishly optimistic).
You know what depiction of AI has held up, and maybe makes even more sense in light of modern AI? Star Wars’ take on Droid. Prior to LLMs it seems obvious that droids were sentient and deserving of rights. Post LLMs… it seems pretty plausible you could get something like C3PO that blabbers on but lacks any meaningful sentience or sapience. Also, it seemed obviously idiotic the way the Trade Federation had humanoid droids acting as pilots and gunners instead of building proper autopilots and targeting system. But after seeing LLMs get shoved everywhere, I totally could imagine a megacorp shoving droids built from standardized pretrained modules into all kinds of applications they aren’t fit for.
We Need To Talk About Leopold | Patrick Boyle yt
My suspicion is that ‘Situational Awareness’ is the most referenced and least finished document in Silicon Valley since the Terms and Conditions.
The essay went viral among tech executives when it was released and Tim Ferrris crowned Leopold the Nostradamus of AI, which is more fitting than Ferris probably intended
New week new guy
It’s depressing how so many (pathetic) software devs are taking their hands off the steering wheel and letting LLMs do all the driving. If you think software quality was poor before LLMs, there’s no way it could have been worse than shit like this:
This became very apparent with GPT 5.6 Sol. It, too, was widely hailed to have near-Fable level intelligence. But I used it non-stop for a week and realized that they had mostly just dialed up the relentlessness meter to eleven, most likely via heavy RLHF.
Last week I gave it a small-sized auth ticket to work on, then stepped away. I came back later that afternoon and found that it had worked for 3+ hours and written 25,000+ lines of code. I skimmed over the code and it looked like a small fix followed by a massive number of additional checks around it, including static analysis tooling.
I gave it to another GPT 5.6 and said “check this code and see if it addresses the ticket”. It looked at it and said that 98% of it was garbage and should be thrown away (its own words). I then gave it to Fable, which said it was massively over-engineered. Fable’s theory was that the agent implemented the fix first, but then compacted and lost crucial context, forgot what the original task was about, and kept going. After many compaction cycles it was completely lost.
On one hand, this is awful. On the other hand, if I worked at a garbage tech company and my boss would not only pay me to sabotage the code base and spend their money but also reward me for doing so… I mean that’s an attractive proposition.
No for real it’s baffling. What happens when your profession doesn’t have a licensing board.
On one hand, this is awful. On the other hand, if I worked at a garbage tech company and my boss would not only pay me to sabotage the code base and spend their money but also reward me for doing so… I mean that’s an attractive proposition.
Before, sabotaging your competition was difficult and legally risky. AI fixes this - now, your competition shall sabotage themselves, leaving you free to focus on other things.
Montana is legalizing a pet Libertarian project: the right to try any experimental medical treatment a crank will offer https://www.technologyreview.com/2026/07/30/1140942/montana-experimental-medical-hub-pushed-forward-right-to-try/ Rationalists are big on this one because they are such good suckers.
VC fundraising pitch: local Dutch Bros knockoff that offers drive-thru coffee enemas
The mechanics of this are left as an exercise to the reader
Hi sir, so that’s 1xArtisinal Coffee Enema with our dark roast single source blend. Says here you turned down the splash proofing upgrade because “the cybertruck is self cleaning, it’s even a boat”. In that case, that’ll be $500. You’re gonna want to drive straight past this next window to the far window. If you could arrive with your pants down and in a crawl position facing away from the window that’d be great.
My new high tech idea. Im offering artisinal reverse coffee enemas, 150 dollars per cup. The specific type of coffee is selected by my advanced Artificial intelligence system:
(Just generate a random name for a plain cup of coffee for them to drink).
Oooh, I hope we get to see a lot of experimental-sounding puberty blocker and hormone replacement trials.
AI companies prob now bulk ordering books to destructively upload from second hand stores.
(Apologies for the strange wording, I was reminded of an Reynolds book where they destroy peoples bodies to upload them and then the uploads all fail).
“monument to the first 500” or something, really fun concept
I kinda like the Revelation Space universe, but I am literally stuck at 85% on book 4 or 5
Company Offering ‘100% Human-Written, Never AI’ Medical Research Is Entirely AI | 404
lol of course it is. Sooner or later, we’ll have to come up with some way to label actual natural language in the marketplace, like certified organic like fruit or something.
The fake research company listed Dr. Elena Vasquez, one of the most commonly made-up AI names, on its staff page. I guess Elara Voss was too busy.
Perhaps we should page Dr Goblin.
Lesswronger argues skynet might still be a decade away… because they’d rather have the eugenics wars instead: https://www.lesswrong.com/posts/iQzxxgJXXaAQjq7Jz/faq-isn-t-agi-coming-too-soon-for-reprogenetics-to-help
why do they keep calling it *genics, as if everyone doesn’t immediately understand what you mean
They have a clause for “Most adult HIA methods (anything involving brain surgery or gene editing) would probably be somewhat hard to scale up a lot; some may be quite easy to scale up (e.g. non-editing brain drugs).”
Something something used surgical equipment in Ziz’ van, something something they do like their pills.
Well you see, if they used a real technical term for an existing genetics technique or technology, their fantasy might be limited by the limits of reality, so instead they user complicated jargon terms to signal they are talking about
fantasy“near future” techniques and technologies.
Comments are utterly mired in jargon
Humanity should count as actual AGI, but it remains slow.
It’s a weird middle ground, where it is fooming but pretty slowly. (I mean, as a human, I would think that, i.e. I would experience time on a scale that’s somewhat faster than humanity’s foom.)
I don’t think the term “FOOM” as ever been explicitely defined, it’s just verbal tic/shorthand for “intelligence explosion”. How can something explode slowly???
also nice save defining “AGI == humanity” => goal reached
I vaguely remember someone saying the term “foom” came from the sound of something blowing up/expanding quickly, could be wrong on that tho
But yeah “fooming slowly” is not fooming. That’s just it going slow
I mainly understand “foom” as a sound effect onomatopoeia from comic books. Which is totally appropriate given the foundational texts of LW
also nice save defining “AGI == humanity” => goal reached
A common game along these lines I’ve seen: define the singularity as knowledge-work inventions allowing for more inventions. Thus we’ve been in the singularity since the invention of the
printing presswritinglanguage.
Alpha is an investing term (whatever performance you can’t explain by the market cycle and known mathematical properties of specific stocks). I feel so tired.
Stealing technical terms from other fields to make their fantasies feel more real.
I regret to inform you, that this garbage was run by The Register a while ago.
It shouts out several friends of the sub, including @[email protected], PivotToAI, OpenSlopware, Robert Kingett & Ludicity.
The point of the piece is that all anti-AI folks should just get along, and if we are to stand a chance against corporate AI we need to get over our petty grievances and reach out a hand to the racists, homophobes and crypto-bros and work together in the spirit of Open Source!
Liam’s here, for what it’s worth.
Torvalds’ position is pragmatic: his interest is in whether it works or not.
But, of course, the chatbots do not work. In a pragmatic sense, they do not produce maintainable integrated code changes with low defect rates. In a societal sense, they do not replace human laborers, despite the delusions of management.
One of the most original and innovative OS development projects of the 21st century so far was Urbit, but it is closely entwined with cryptocurrencies. Urbit’s original creator, Curtis Yarvin, has reportedly espoused some extreme beliefs; The Nation called him The Reactionary Prophet of Silicon Valley.
Wild way to admit no knowledge of Arcan, Fuchsia, etc. Honestly, even a basic overlay network like Yggdrasil can win an apples-to-apples comparison with Urbit. I feel like this is an instance of critihype for fascist weirdos; similar stuff gets said about Justine Tunney, another cryptofascist developer whose projects are sometimes silly and weird. Doubly weird for those of us who know about Urbit’s internals; Urbit originally was not a cryptocurrency project and it used to have non-Yarvin/Tlon forks.
It’s an overly simplistic way to reduce a complex and nuanced situation, but one way to consider this is in terms of pro-AI and anti-AI, versus “woke” and “anti-woke.”
More seriously, add a third dimension, a Butlerian dimension, to the political compass: to what degree may your automated devices appear to be human? One extreme is Asimov-style transhumanism and the other extreme is Luddite loom-smashing. However, recognition of this dimension doesn’t negate the other dimensions and we shouldn’t work in cooperation with fascists.
One of the most original and innovative OS development projects of the 21st century so far was Urbit
So innovative, it even has a fandom!
https://archiveofourown.org/tags/Urbit
(snerk)
Liam’s here, for what it’s worth.
I almost wrote down, it sounded like it was written by a lurker on techtakes. The overlap of takes was just so damn high.
More seriously, add a third dimension, a Butlerian dimension, to the political compass: to what degree may your automated devices appear to be human? One extreme is Asimov-style transhumanism and the other extreme is Luddite loom-smashing.
The luddites smashed looms not because they were inherently anti-technology, but as a means of industrial action in a time when labour held no formal power at all. As you say yourself, the issue with AI is that it doesn’t even work as advertised; industrial looms worked fine. «AI»'s biggest problems are the enormous waste, ecological damage, enshittification and labour discipline that it incurs. If Silicon Valley was actually building Data, the debate would be different, but this dimension seems mostly theoretical for now.
Urbit’s original creator, Curtis Yarvin, has reportedly espoused some extreme beliefs
bro why do you write like this
“reportedly”, a german word meaning “this author couldn’t be arsed to go looking”
I mean I’m more than happy to extend grace to randos, but if you’re doing journalism, come on. He’s not just a guy with opinions! He’s a leading far right intellectual!
His Wikipedia intro tells you more than enough with no “allegedly” in sight.
yeah, that’s about where I land usually too
I guess the thing is that I know how much “shithead with a column” has become possible so I don’t really, off the cuff, know how journo most journos are
(I shall leave aside for the moment the entire tirade this makes me want to go on)
Yech, the comments…
you made me look
But I feel that as time moves on, the level of future shock will subside
fuck of a thing to “just” shrug all the criticism off simply as future shock
they’re taking incorrecting each other to whole new levels
My unrealistic sicko fantasy: that Torvalds pulling the big ol slop lever drives enough oppositionally defiant (positive) people away from Linux development that alternative OS devel gets a serious boost.
A question for the wise: what comes after Linux?
Vindicate Tanenbaum in the greatest Usenet thread in os development: make it a microkernel.
Somebody writes a low-level kernel for Emacs and the revenge of the Lisp machines begins
maybe, but SBCL succumbed to the siren song of slop, who knows what will be next.
Yeah, seeing that gave me the black pill. I chose Common Lisp for personal projects to avoid techbro bullshit.
lisp went big in the leadup to the AI winter, which followed a lot of fraudulent nonsense that even the DoD would only buy for so long
so in that sense it’s in character
eyes copy of ANSI Common Lisp written by Paul Graham
Picard frowning at the mirror universe edition of the bible
That article is a ray of sunshine
plan9 on riscv(or sparc or subleq or whatever) is also truly what I want for the future. It obviously won’t be what happens(most likely linux will stay, maybe one of the less slopped bsds will take over) but please gods please
Would be cool if Redox os gets more popular, they have an explicitly anti-LLM policy. Microkernels are neat, and I enjoy Rust. I think it’s an unfortunate truth that everything in the future will still have to be Unix-y with compatibility layers to support existing software.
This looks rad as hell. I had no idea Redox was so far along!
I want it to be plan 9.
We are not worthy
The irritating part of all of this is that after 20+ years we finally have a (very) few companies that sell machines with Linux pre-installed. The only thing I ever learned from installing OSes is that I hate installing OSes.
I would love to boycott all software that accepts garbage [1] contributions, but I do not think that is practical. I think I might have to settle with just refusing to use it myself, and wait until everyone realizes that gen AI is just a bad idea.
Note 1: Satya Nadella said he did not want people referring to AI as “slop”, so I will grant his request and refer to it as “garbage”.
Realistically a BSD
I am in fact moving back to NetBSD, but the ambiguity of their LLM policy makes me wary/tired.
My only hope is that NetBSD moves so slowly anyway that the devs’ traditional preference towards clean, minimalistic readable code above all else holds off any “slop commits verified by maintainers” long enough that the bubble pops before slop gets any of its clawdes in core code at least.
RiP Old! SneerClub which is now behind a login wall on Reddit. Fortunately we backed up its earliest posts. I don’t plan to check the site regularly.
I see elsewhere on lemmy that it’s old reddit that gets loginwalled, not sneerclub specifically
either way still works for me
Seems like reddit.com is not behind a login wall yet, but give them time.
For a while now I’m being randomly login-walled on reddit.com on varying subreddits, in ways that seem nondeterministic. I hope they login-wall the entire thing, I’ll finally ditch my addiction then
on balance i’ve been banned on various sites that think i’m not human enough, permanently from mouser and intermittently from a couple of others including startpage (one day total). i guess i’m catching strays because of residential proxies. this must be this disruption ai boosters were waiting for
Rss to subreddits still work just case anyone cares
Any reason why?
Officially old.reddit.com is closed to reduce costs from scrapers at chatbot companies. Really so they can collect even more personal information from every visitor.
Thanks
til amodei’s married, and his wife is among the top advisors. she also has a rather stunning set of choices, including seeking money for a porn site (with what in summary appears to be a sensible goal) from epstein after his conviction
I knew Amodei was married but holy SHIT I did not know that
that article is rather a few interesting curveballs
And of course our favorite polyamorist turns up:
The most important would turn out to be Amodei, whom she started dating in 2014. Just previously, beginning in 2011 and lasting for three years, she was in a relationship with Schmidt, who was then executive chairman of Google.
Definitely seems like an echo of Altman’s path to influence. If your “meritocracy” keeps empowering these odd mini-Rasputins, where’s the merit?
the merit seems to be stored in the balls
Amazon’s latest planned data centre is gonna be putting out as much emissions as an entire country
I cannot express how badly this AI bullshit needs to go
worse yet, Amazon actually has uses for that much data centre
