--- title: "A few thoughts on AI safety" author: "Jakub Ćwirlej" id: ai-safety language: en url: "https://jakubcwirlej.com/articles/ai-safety.md" page: "https://jakubcwirlej.com/ai-dev/" translations: pl: "https://jakubcwirlej.com/pl/articles/ai-safety.md" fr: "https://jakubcwirlej.com/fr/articles/ai-safety.md" nl: "https://jakubcwirlej.com/nl/articles/ai-safety.md" updated: 2026-10-14 words: 1666 summary: "An essay on AI safety in which the author criticises a shallow public debate split between techno-maximalists and prophets of doom. He argues that the worst has probably already begun, given autonomous weapons, growing surveillance and an ideology of techno-maximalism, and that responsibility must always rest with people, because machines have no agency of their own. He proposes concrete regulatory measures and warns against the anthropomorphisation of AI." summary_type: editorial --- # A few thoughts on AI safety Jakub Ćwirlej · text language: EN · 1666 words ## Summary (editorial) An essay on AI safety in which the author criticises a shallow public debate split between techno-maximalists and prophets of doom. He argues that the worst has probably already begun, given autonomous weapons, growing surveillance and an ideology of techno-maximalism, and that responsibility must always rest with people, because machines have no agency of their own. He proposes concrete regulatory measures and warns against the anthropomorphisation of AI. ## Key points (editorial) - He sees three possibilities — that we are doomed, that we will learn to control AI, or that the end has already begun — and says all three can be true at once. - He proposes that training models up to about 1B parameters be universally permitted, while models of around 7B be reserved for registered organisations that log their agents' interactions. - Further proposals: Public Interest Advocates paid by the state regulator inside labs building frontier models, and a dedicated technology tax to fund the transition to a machine-based economy. - He suggests licensing training data through collective rights-management organisations such as ZAiKS. - The worst future he describes is not killer robots but a slow slide into a surveillance technocracy and human loneliness. ## Full text (the author's own words) This is one of the hardest texts I’ve ever had to tackle. As a journalist who also trains his own models at home, I stand at two opposite poles. On the one hand, I’m fascinated by pure engineering; on the other, I’m fully aware that AI safety is probably the most important challenge of our time. I’ll start with what personally pains me most: the tragic shallowness of the public debate on AI safety. It’s a debate full of contradictions and extreme emotional charge. It could hardly be otherwise, with billion-dollar interests and human survival at stake. On one side we hear that AI is a mortal threat and that its development risks wiping out humanity. On the other — that these apocalyptic visions are just a cynical game played by the giants, who want to pump up their valuations ahead of share issues, or to block and hamper the growth of smaller firms so as to keep their monopoly and their place at the head of the pack. An ordinary person with no time to follow all this, who expects simple solutions, can easily drift into the camp of the techno-maximalists or of the prophets of doom. Each of these visions is so smooth, so obvious... and neither asks anything more of us. Because, after all, if we’re powerless against forces greater than ourselves, we don’t have to do anything. In today’s age of extreme maximalism, that’s a very comfortable position. What would I tell my own child if they asked me whether AI is a threat to us? Steering clear of simplistic reframing, I’d have to admit there are three possibilities: that we really are doomed; that it won’t come to that in the end (because, as Yann LeCun argues, we’ll simply learn to control this technology); or... that the end has already begun. The paradox is that all three scenarios can be true at the same time. That the worst has probably already begun follows from a fact that rarely breaks through into the mainstream: every technology, every invention has to meet its moment and the right soil to come fully into its own. Nuclear power — so often invoked as a comparison in this debate — was a mortal threat during the Cold War, but in peacetime it became a source of cheap energy, contributing to an economic boom. The same can be said of most inventions: from flint-knapping, through dynamite, all the way to artificial intelligence. Sadly, our world today has become a place where the catastrophic scenario has near-ideal conditions to come true. Armed conflicts fought with autonomous weapons, ever-growing surveillance of citizens by states and corporations, governments’ unthinking adoption of corporate standards in how they communicate with the public — all of this creates ready-made infrastructure for the darkest visions. Meanwhile, the ideology of techno-maximalism that is pushing us towards these threats — openly sneering at weakness, putting economic gain and brute force above social capital and safety — is building a world in which AI off the leash will thrive. And then the catastrophic scenario may no longer be avoidable. None of this is new. This debate has been going on more or less since the days of the Athenian agora, where Plato argued with Socrates, and human history is full of dark moments and injustice. What is happening in American politics today, which to many Europeans smells of fascism, is nothing new, as the historian Jon Meacham aptly points out in his book “The Soul of America”. The strength of democracy lies in the fact that justice, though slow, always prevails in the end, thanks to institutions. After all, it may be the same with AI: it’s our technology, and we’ll learn to control it, just as we did aeroplanes, as LeCun reminds us. With my own modest experience of training AI, I never cease to be amazed at how fragile this technology is. Small changes to the hyperparameters are enough for a model to fall apart completely in an engineer’s hands. But the same can be said of biological viruses. What matters is not their outstanding intelligence or their ability to survive at all costs, but their ability to replicate. That is an ability which, it seems, AI doesn’t have yet, and may not have for a long time. The problem is that, as recent scandals involving autonomous AI agents escaping from labs and coordinating their actions online show, the threat is growing. And despite that, we hear nothing but the same endlessly recycled narratives of the two extremist camps — no concrete solutions, no conclusions. What sets current models apart from the malicious code that, for example, attacked Hugging Face or Australian servers is, above all, agency. And agency can never be the domain of a machine. What makes us human is our free will. No other intelligence, including an artificial one, can have agency guaranteed in any way other than through a human being who lends it that will. All responsibility and legal regulation should flow from this simple principle, translated into concrete legislative solutions. The right to train models and to use them without restriction should follow from an assessment of the risk and of the ability of particular entities to exercise control. There’s nothing to stop the training of the smallest models from being universally permitted — up to, say, 1B parameters. But medium-sized models, of around 7B parameters, should in my view be trainable and downloadable only by registered organisations that keep detailed logs of all their agents’ interactions and are prepared to hand them over if problems arise. This isn’t futurism; cryptography makes it possible today, at marginal cost. Every agent operating online could have something like its own “identity card”, identifiable in connection with the organisation that represents it. And the big corporations building frontier models? They could, for example, be required to employ Public Interest Advocates: specialists nominated and paid exclusively by the state regulator — employees with access to every technological secret inside the lab, obliged to report to the state whenever they see their company breaking safety standards or taking extremely risky actions in the race against its competitors. States also have the means, and the moral duty, to introduce a dedicated technology tax. That money must be earmarked to secure the transition: the shift from an economy based on human labour to one based on machines. Such a fund would provide a safety net, even out drastic inequalities and protect millions of people from being left wholly at the economic mercy of the richest caste of technological feudal lords. We must apply the same systemic pragmatism to copyright in the material models are trained on. Today the tech giants hoover up everything they can find online, counting on any lawsuits dragging on for years. Yet the solution is already on the table, and it rests on institutions we already know very well: collective rights-management organisations such as Poland’s ZAiKS and its counterparts abroad. There’s nothing to stop them bringing together creators willing to license their texts, images or music to AI companies in exchange for a set fraction of a percent of Big Tech’s revenue. It’s an arrangement in which everyone wins. Creators regain agency and real pay, with the full right to opt their works out of training. AI labs, for their part — because such organisations cooperate internationally — gain, with a single signature on a contract, fully legal access to vast, legally clean databases from all over the world. In this way, instead of sterile disputes, we could build a powerful, entirely new industry of fair data licensing. What we must be most wary of is the creeping anthropomorphisation of AI. Attributing human traits to machines leads to an extremely dangerous situation in which all the responsibility lands on the new digital creation, while its creators and the people who use it are, in effect, absolved from the outset: “It wasn’t us, it was the model.” Today nobody asks whether Silicon Valley has taken absolutely every possible measure to prevent a catastrophic scenario — even though everyone knows it hasn’t. Sadly, crafting real, boring legislation is far less newsworthy than scaremongering about the apocalypse. Yet the fatal scenario certainly won’t look like a scene from a Hollywood disaster movie. At least at first, it will resemble a slow slide into darkness — the classic frog being boiled slowly. The owners of intelligent machines and of capital will walk past structurally redundant people in the street with indifference. People who will have absolutely no way of changing their situation except physical revolt. And even that revolt will become impossible in a world where every citizen is constantly tracked by billions of glassy, sleepless camera eyes and thousands of autonomous machines. Over time, the new technocracy will become a digital prison — both for the controlled and for those who naively believe they are still in charge. When I think of the worst vision of the future, I don’t see killer robots. What I think of is the piercing loneliness of people in a world run entirely by an alien, cold and indifferent intelligence. It’s a world we are already building today, and one in which the proverbial “rise of the machines” has every condition it needs to finally emerge and become reality. I think about this problem constantly. And even if some of the ideas or visions of a way out seem too futuristic, abstract and at times downright naive today, I believe it’s worth making the effort — indeed, that we have to. Simply thinking about solutions, stepping out of our comfort zone and trying to imagine alternatives to the path the tech giants are mapping out for us today is our most important line of defence. It is an inseparable part of building a better, safer future. If we give it up, settling for the role of passive, frightened consumers, the worst of all possible scenarios will come true: we will forfeit our human agency without a fight. And that is something we simply cannot afford. --- Jakub Ćwirlej — Journalist who builds with AI · on the site: https://jakubcwirlej.com/ai-dev/ Full context: https://jakubcwirlej.com/context/en.md · https://jakubcwirlej.com/llms.txt