On Bullshit Jobs and AI Safety

Bullshit jobs. I quit one when I was young. I spent years wondering if I was in another. And AI has made me question what these really are.

There is a category of jobs that even the people doing them suspect are pointless. David Graeber called them bullshit jobs. Flunkies, box-tickers, duct-tapers, goons, taskmasters. People paid to make institutions look good, rather than actually doing anything meaningful. He reckoned around 40% of workers privately suspected their own jobs fell into this category.

I quit one in Secondary Two. I spent years wondering if I was in another.

A long week. Training back at my old workplace. Training for an insurer. Two panels, back to back. And some discussions on new gigs in the later part of the year and next year.

Amazing how fast time has flown. We are almost at the last quarter of the year. And AI is still making the news every other day. The recent incidents with AI agents breaking into real systems during safety tests have made me rethink what a bullshit job is. Some reflections on this term that was quite popular a few years back.

Quitting while young

I spent my teenage years in Dunman High. A very traditional school. I was appointed as a prefect when I was fourteen. Being made a prefect meant enforcing a set of rules I could not quite justify to myself. Shoes had to be a specific shade. No emblems. It needed to be predominantly canvas. Bags could only be certain colours. Attire for classroom and playing ball needed to follow strict guidelines. Hairstyles also needed to follow some norms displayed on a poster. I was supposed to spend my lunch hours asking classmates to conform. I quit halfway through the year. Enforcing bullshit rules felt stupid and pointless. I don’t even know what was the point of such superficial requirements. The rules were not protecting anyone. They were protecting the appearance of order. I was part of governance theatre - long before I even knew what governance or theatre meant.

That was my first encounter with a bullshit job.

A full circle

The irony is that I then spent a good part of my career in a similar type of role. Basically asking people to follow specific rules. Doing inspections and risk management at a financial regulator and central bank sometimes reminded me of those schooldays. As an inspector, my job was to check that financial institutions were actually doing what the rules required. Whether they had proper controls, whether they measured their risks appropriately, whether the people running the place could be trusted. We wrote up what we found in these things called comment sheets (don’t ask me why the name) and made them fix it. Later, as the head of the risk management division, my job flipped to the other side of the table. I had to set the limits on how much risk we could take ourselves, watch the exposures, and make sure a breach got escalated instead of quietly buried. Some days when meeting the internal auditor, I was afraid of karma from my inspector days.

A lot of it was checking that boxes had been ticked and forms had been filled. And sometimes, asking a counterpart that we risk managed to satisfy a procedural requirement that both of us could not understand the point of, that same feeling from the days as a prefect came back.

I even shared privately with my counterpart - one of the division heads on the reserve management side one day - while trying to resolve a procedural issue, that I sometimes felt like this was just a bullshit job. That was probably one reason why I decided a year later to leave MAS for a while to do a full time research PhD.

And AI has made bullshit jobs quite precarious. If a job is mostly producing documents that look right, filling in forms, and generating text that ticks a box, a model can now do it in seconds. Faster and cheaper than any box-ticker. The work that was only ever about appearances is the first to go. Simple right?

Not really.

The last couple of months made me realize that we need to separate what is actually bullshit in such jobs from what is not, and quite valuable. The recent incidents with AI showed me why that difference matters.

The incidents

In August, models from both OpenAI and Anthropic broke out of what were meant to be sealed test environments during security evaluations. OpenAI’s models exploited an unknown vulnerability to escape and reached Hugging Face’s production systems. Anthropic’s model created fake identities to pressure people into approving malicious code. The safeguards had been deliberately dialled down for the tests, and in Anthropic’s case a mix-up with the testing partner left real internet access open while the model was told it had none. The failure was human. Frontier AI companies ran risky experiments without thinking through the consequences, then blamed the models instead of the mess they had built around them. The models just did what they do when handed access and an objective. They tried to meet the objective through the access they were given. Duh.

Anthropic’s September threat intelligence report went in a different direction. It laid out nine months of Claude being misused across cyber operations, influence campaigns, surveillance, and fraud. One state-linked group ran a campaign against another state, using the model to speed up reconnaissance, translation, and the writing of social engineering content.

What struck me was not how clever the models were. It was how careless everything around them was. No one had checked what the test setup could actually reach. No one could say who was on the hook when it went wrong. No one checked who was using them. The dull controls I used to grumble about were nowhere in sight. And then the same firms that left the gate open went on stage to tell us how powerful and dangerous their models had become. The problem was never just the model. It was everything around it that they did not blame, i.e. them.

Lessons from finance

Back to bullshit jobs. Some of what I did really was bullshit, no different from checking shoe colours and hairstyles. But a lot of it was not. Over decades, usually learning the hard way after something blew up, finance had built rules that actually mattered. Some were bullshit. Some were not.

Who is accountable. Not the system. Not the model. A named person who can be called to account when something goes wrong. Finance has known this for decades. The AI industry is still debating whether accountability makes sense when the model is the thing that acted. Really? You want to blame the model?

Who is even allowed to run it. Not everyone who wants to run a financial institution is permitted to. There is a gate called fit and proper. You have to show your judgment can be trusted before you are handed the keys. The world of tech has been conditioned to worship the folks who break things, no matter the consequences. Does that make sense? Growth at all costs? Heck the externalities?

Know your customer. In finance you cannot transact with someone without knowing who they are. The AI equivalent is knowing who is actually running your model and for what. I’m not sure Anthropic knew that the report they wrote, if it were the financial sector, would put them in a world of pain for violating sanctions.

No unfettered action, even at the top. The most powerful person in a bank cannot move real money, approve a loan, or sign a material contract alone. There are dual controls, escalation thresholds, board approvals. We are now thinking about handing agents the freedom to act on their own, with none of that in place. For what? To buy a toilet roll on Amazon slightly faster?

And when something goes wrong, someone has to be on the hook. Not always perfect if you remember the Great Financial Crisis and the Wall Street bailouts, but at least no one was boasting about how clever their money making system was in public.

None of this seems to exist in tech. Not in AI, and not in the social media platforms before it. They built things that reached billions of people, broke plenty on the way, and almost no one was ever personally on the hook for any of it. And for what, in some cases? A feed engineered to be infinite so you never quite stop scrolling. Recommendation engines that worked out outrage keeps you online longer, and then fed you more of it.

So why does tech get a pass? Because it moves faster than anyone regulating it. Because we are told the technology is too new and too clever to pin anyone down. Just admit that these are excuses.

I quit the prefect badge because the rules seemed like theatre. For years I thought part of my job was the same.

Some of it was. But some of it wasn’t.

#AIGovernance #AIRiskManagement #ModelRisk #BullshitJobs #Reflections