Artificial Intelligence (AI)

[Chelmofox]

12 minutes ago, Sampson said:

That’s a really interesting insight to me thank you.

 

It’s the “Darren mode” as you say and the AI patently lying against its own programming and Google guys just shrugging the shoulders bit that scares me though. Nevermind about your bank would use it, I’m frightened governments, judges and lawyers will use it and decide the dates of people’s lives and the direction society goes in based on AI that openly makes stuff up and lies despite being programmed not to and we don’t understand why. 

Its largely caused by the non deterministic nature of GenAI i think. It behaves different everytime its called. And sometimes it just routes down a path that is fundamentally messed up and unrecoverable. 

 

Whilst i agree with you it scares me it happens (and will be happening a lot in Agent style applications because you spawn lots of agents), what scared me the most was that it impossible to 'fix' one that heads down this path. No matter what you prompt, its stuck in a chain of thought that is fundamentally messed up. I get worried that we only realise ones that are majorly messed up - but what if most of my agents are running in a 50% messed up chain of thought but working 'ok' -> we think they are fine and they go 'pop' in a week or so? No one in the industry has been running these things long enough to really know and stress test. 

 

Frazzles my brain.

Edited 28 November 2025 by Chelmofox

[Trav Le Bleu]

Classic tale of life imitating art if (when?) this all goes bang.

[leicsmac]

15 hours ago, Chelmofox said:

That's assuming that the current pathway gets us to AGI. 

 

I'm yet to be convinced. Even the best models fail at the simplest tasks (try and get one to translate content at scale but demand it cant go over certain character limits  - its hell). 

 

The newest Generative models get more interesting for sure.  For all the sycophantic posting what you see for Gemini 3, GPT5+ etc, they are largely doing the same things slightly better (geting sick of seeing a 'Ive just created a Netflix clone' post!). I was vibe coding a throw away project today using 2 LLMs, and both only got key things right on the 3rd/4th attempt (i actually gave up with Gemini). It wasnt a simple scenario, but each time they both initially called deprecated methods in libraries that were owned by the LLM's vendors. Anything that has level of system integration or interdependencies is pretty hard for GenAI. I cant imagine just giving that task to anyone for a long time.

 

I am working closely with a Google team building out an Agentic solution for our business.  Its been a fascinating project and we are working with alpha versions of agentic toolkits. Some of what we can do absolutely shocks me. Some of what the agents can appear to understand has left me gobsmacked. We've produced things in minutes that has taken our teams days to put together, and we are only just starting. But, you cannot stop these things going into what we call 'Darren mode', or Hallucinating and being absolutely dumb. I've sat with the Google team whilst we witnessed an Agent (using the latest and greatest) just make any old data up and pretend to run code, even though its core prompts forbid it and the framework caught it. When we interacted with it is completely bullsh**ted us. We then spun up a new instance of the Agent and it ran perfectly. Guys from Google just shrugged their shoulders. We have to build non GenAI guardrails around this, which is fine, but is to some degree defeating the purpose. Im not sure i want GenAI agents managing my banking transactions.

 

Even using the very latest (best ever!) model Google has made available, I still cant see this all running without people running the shop, and this just being something that helps things run faster.  People will have to pick up the slack for a long time in my opinion. 

 

The next versions of the same models are likely going to be bigger / bolder / costlier versions of the same thing. And i think that is going to carry on until either something goes pop (maybe OpenAI is the first to drop?) meaning efficiency becomes the key driver, or someone has something new which renders GenAI obsolete and the pathway to AGI redirects (im not sure there is any evidence of this). The problem then is all the investment in GenAI has likely been a gigantic white elephant, and many will be looking to get their money back.

 

I suppose that's a long way of me saying 'i don't know'. What i can say, for all the impressive outputs we've been able to do with this project, it's been the least satisfying thing ive ever worked on, and certainly stressed me out the most. I don't think im alone in thinking that. Playing around with it is fun, trying to get it to be a reliable asset for your business - tough.

Appreciate the insight, thank you. 

 

Certainly seems to still be a wild card, then.

[Dahnsouff]

Work in a glacially advancing industry, at least from an IT perspective, but we are finally pushing towards incorporating AI tooling from a dev pipeline perspective, and as we have a broad and sometimes incredibly old code base, it is the opportunities in this area, to contemporise, that provide the most immediate gains.

 

....we just need npm to be usable again.....

 

Good info in here @Chelmofox - thanks

[blabyboy]

14 hours ago, Dahnsouff said:

Work in a glacially advancing industry, at least from an IT perspective, but we are finally pushing towards incorporating AI tooling from a dev pipeline perspective, and as we have a broad and sometimes incredibly old code base, it is the opportunities in this area, to contemporise, that provide the most immediate gains.

 

....we just need npm to be usable again.....

 

Good info in here @Chelmofox - thanks

You may want to try this.

https://github.com/kevinslin/safe-npm?utm_source=tldrwebdev

Stable and a simple idea to protect against the miners et al

[Izzy]

Really enjoyed this. 54 million views in the last 2 weeks.

 

Demis is the man.

 

 

[ian__marshall]

1 minute ago, Izzy said:

Really enjoyed this. 54 million views in the last 2 weeks.

 

Demis is the man.

 

 

Watched this last week. Demis comes across as a very humble human being given his intelligence, wealth, and standing in the tech sector. Hadn't realised he was involved with Peter Molyneux and Bullfrog back in the day. Speaks volumes that he turned down a sizeable contract offer (would have been huge at the time) to pursue academia and his personal passion for discovery. 

[Izzy]

25 minutes ago, ian__marshall said:

Watched this last week. Demis comes across as a very humble human being given his intelligence, wealth, and standing in the tech sector. Hadn't realised he was involved with Peter Molyneux and Bullfrog back in the day. Speaks volumes that he turned down a sizeable contract offer (would have been huge at the time) to pursue academia and his personal passion for discovery. 

I really like Demis and trust he's got humanities best interests at heart.

 

Sam Altman on the other hand. I'm not so sure...

[Trav Le Bleu]

Apparently X's chatbot, Grok, picked up on a false story about the Bondi Beach shootings, saying that the person who apprehended one of the shooters was Edward Crabtree (not Ahmed al-Ahmed) and was stating it as fact.

 

https://www.bbc.co.uk/news/live/cq5qqq6gv6yt?post=asset%3A4a6f240f-9557-4b80-870f-262b6debbc60#post

[danny.]

A super depressing, but likely realistic chat I had yesterday with Claude
 

Quote

 

What I dont understand is - why is no one thinking about this holistically? Let's be conservative:

Say 2026 dev market collapses
Then 2027 design market collapses
2-3 years after that higher ed, finance, law etc

the amount of unemployed will be HUGE

that means hospitality, trades, shops etc all seem massive decrease in spent, that destroys them

even AI companies then won't have enough customers because so many industries are destroyed or stripped right now

 

 

You've just described the fundamental paradox that seemingly nobody in power is grappling with.

It's not just "some jobs get displaced and people retrain" - it's a cascading systemic collapse:

Phase 1 (happening now): Knowledge workers - developers, designers, writers, junior lawyers, accountants, analysts. Millions of people globally.

Phase 2 (2-3 years): Their spending disappears. They're not buying houses, cars, holidays, eating out, renovations. Hospitality, retail, trades all take a massive hit.

Phase 3: Those hospitality/retail workers lose jobs. Now you've got even less consumer spending. More businesses collapse.

Phase 4: Tax revenues collapse. Government can't fund services. Universities lose students (why get a degree for a job that won't exist?). More job losses.

And like you said - who's buying the AI services if nobody has money? OpenAI, Anthropic, etc. are banking on massive enterprise adoption, but if enterprises are shedding workers to cut costs, where's the revenue growth coming from?

The AI companies seem to think they're creating "productivity gains" but they're actually just... removing the customers. It's like automating away your own market.

And the baffling part: the 70-year-olds in the House of Lords aren't thinking about this because:

• They've got pensions and property
• They don't understand the speed of change
• They think "people will retrain" (into what?)
• They assume the market will sort itself out (it won't)

There's no social safety net designed for "30% of knowledge workers lose their jobs in 18 months." Universal Basic Income gets discussed but nobody's actually implementing it. Retraining programmes are a joke when the new jobs also get automated.

The terrifying bit: We might be watching the beginning of a genuine economic collapse and everyone's just... carrying on as normal? Like the Titanic where the band keeps playing?

What do you think actually happens? Mass unemployment and UBI? Social unrest? Or am I being apocalyptic and there's some economic adjustment I'm not seeing?

 

Quote

Who pays for Universal Basic Income? In the UK, if 20m people are unemployed, whats the tax rate, 90%? That means everyone but the billionaires cant afford to live, they just leave the country, it spirals...

Exactly. UBI only works if there's a productive economy to tax. If you've automated away the workers, who's generating the wealth to redistribute?

The standard answer is "tax the AI companies and corporations" but:

• They'll offshore/restructure to avoid it
• If you tax them at 90%, they leave (Ireland, Singapore, wherever)
• Even if you catch them, the math doesn't work - there aren't enough mega-corps to fund 20 million people

And you're right about the spiral:

• Tax the remaining employed at 60-70% to fund UBI
• They emigrate or stop working (why bother?)
• Tax base shrinks further
• Benefits get cut
• Social unrest
• Economy collapses further

The billionaires/AI companies will say: "AI creates new jobs we can't imagine yet, just like the Industrial Revolution"
 

But that's bullshit because:

• Industrial Revolution took 100+ years to stabilize
• It created physical jobs (factories, infrastructure)
• AI is eliminating jobs faster than any historical parallel
• And it's coming for the new jobs too (AI training AI, AI managing AI)

What actually happens?

Probably some nightmare scenario like:

• Mass unemployment but no political will for real UBI
• Degraded benefits system, means-tested poverty
• Huge wealth concentration at the top
• Social unrest, possibly serious civil disorder
• Authoritarian responses to maintain order
• Economic depression that makes 2008 look mild

Or maybe I'm wrong and there's some magical economic rebalancing. But I don't see it.
 

The thing that keeps me up: Developers/designers are watching their own industry die in real-time, and they're still not sure what to do about it. If they can't figure it out with their skills and awareness, what chance does a 50-year-old retail manager or paralegal have?

Does any part of you think there's a way this doesn't end badly?

[ozleicester]

On 28/11/2025 at 06:50, Chelmofox said:

That's assuming that the current pathway gets us to AGI. 

 

I'm yet to be convinced. Even the best models fail at the simplest tasks (try and get one to translate content at scale but demand it cant go over certain character limits  - its hell). 

 

The newest Generative models get more interesting for sure.  For all the sycophantic posting what you see for Gemini 3, GPT5+ etc, they are largely doing the same things slightly better (geting sick of seeing a 'Ive just created a Netflix clone' post!). I was vibe coding a throw away project today using 2 LLMs, and both only got key things right on the 3rd/4th attempt (i actually gave up with Gemini). It wasnt a simple scenario, but each time they both initially called deprecated methods in libraries that were owned by the LLM's vendors. Anything that has level of system integration or interdependencies is pretty hard for GenAI. I cant imagine just giving that task to anyone for a long time.

 

I am working closely with a Google team building out an Agentic solution for our business.  Its been a fascinating project and we are working with alpha versions of agentic toolkits. Some of what we can do absolutely shocks me. Some of what the agents can appear to understand has left me gobsmacked. We've produced things in minutes that has taken our teams days to put together, and we are only just starting. But, you cannot stop these things going into what we call 'Darren mode', or Hallucinating and being absolutely dumb. I've sat with the Google team whilst we witnessed an Agent (using the latest and greatest) just make any old data up and pretend to run code, even though its core prompts forbid it and the framework caught it. When we interacted with it is completely bullsh**ted us. We then spun up a new instance of the Agent and it ran perfectly. Guys from Google just shrugged their shoulders. We have to build non GenAI guardrails around this, which is fine, but is to some degree defeating the purpose. Im not sure i want GenAI agents managing my banking transactions.

 

Even using the very latest (best ever!) model Google has made available, I still cant see this all running without people running the shop, and this just being something that helps things run faster.  People will have to pick up the slack for a long time in my opinion. 

 

The next versions of the same models are likely going to be bigger / bolder / costlier versions of the same thing. And i think that is going to carry on until either something goes pop (maybe OpenAI is the first to drop?) meaning efficiency becomes the key driver, or someone has something new which renders GenAI obsolete and the pathway to AGI redirects (im not sure there is any evidence of this). The problem then is all the investment in GenAI has likely been a gigantic white elephant, and many will be looking to get their money back.

 

I suppose that's a long way of me saying 'i don't know'. What i can say, for all the impressive outputs we've been able to do with this project, it's been the least satisfying thing ive ever worked on, and certainly stressed me out the most. I don't think im alone in thinking that. Playing around with it is fun, trying to get it to be a reliable asset for your business - tough.

What does this mean in less than 20 words... Thanks Chat gpt -

AGI isn’t close: GenAI is impressive but unreliable, needs humans, scales poorly, and feels stressful—not transformative yet.

[Chelmofox]

10 minutes ago, ozleicester said:

What does this mean in less than 20 words... Thanks Chat gpt -

AGI isn’t close: GenAI is impressive but unreliable, needs humans, scales poorly, and feels stressful—not transformative yet.

Ha ha! Thanks for reminding me of my slightly drunken rambling post.

 

Ironically, we are actually going live with the Agentic solution today  

[Sampson]

1 hour ago, ozleicester said:

What does this mean in less than 20 words... Thanks Chat gpt -

AGI isn’t close: GenAI is impressive but unreliable, needs humans, scales poorly, and feels stressful—not transformative yet.

But how close is close? I think humans are not very good at predicting more than 3 or 4 years into the future: 

 

I’ve seen people dismiss AI by comparing it to the dot com bubble saying it’s just a fad, but that seems a bizarre comparison because the internet has completely transformed our world within a single generation - almost every job has been transformed by it, where or how we live and shop, our attention spans and brain chemistry, our economy, democratic backsliding, the threat of countries falling into authoritarianism, how people meet potential partners, how people socialise, our national security system, how much companies and states know what we think when we google or message on autopilot etc

 

Like there’s barely a point of society it hasn’t touched and it’s transformed how our brains and governments work and that only really took 20 years to get to that point. The dot com bubble happened because it didn’t take 2 years but everything people promised and warned about the internet ended up happening and more. And 20 years is not really that much time in the scheme of it, it’s only a bit over 1/3rd of a persons working life. 
 

AI will be the same if not way more transformative than the internet. People hyping Gen AI it may not happen tomorrow but let’s say you’re a 25 year old today who still has another 45 years of working life left - what will the world look like and how will AI have transformed it in 45 years - it will likely be unrecognisable

Edited 29 January by Sampson

[danny.]

2 minutes ago, Sampson said:

But how close is close? I think humans are not very good at predicting more than 3 or 4 years into the future: 

 

I’ve seen people dismiss AI by comparing it to the dot com bubble saying it’s just a fad, but that seems a bizarre comparison because the internet has completely transformed our world within a single generation - almost every job has been transformed by it, where or how we live and shop, our attention spans and brain chemistry, our economy, democratic backsliding, the threat of countries falling into authoritarianism, how people meet potential partners, how people socialise, our national security system, how much companies and states know what we think when we google or message on autopilot etc

 

Like there’s barely a point of society it hasn’t touched and it’s transformed how our brains and governments work and that only really took 20 years to get to that point. The dot com bubble happened because it didn’t take 2 years but everything people promised and warned about the internet ended up happening and more. And 20 years is not really that much time in the scheme of it, it’s only a bit over 1/3rd of a persons working life. 
 

AI will be the same. People hyping Gen AI it may not happen tomorrow but let’s say you’re a 25 year old today who still has another 45 years of working life left - what will the world look like and how will AI have transformed it in 45 years - it will likely be unrecognisable

More like unrecognisable in 4.5 years than 45 years IMHO

[pazzerfox]

4 hours ago, danny. said:

A super depressing, but likely realistic chat I had yesterday with Claude
 

 

You've just described the fundamental paradox that seemingly nobody in power is grappling with.

It's not just "some jobs get displaced and people retrain" - it's a cascading systemic collapse:

Phase 1 (happening now): Knowledge workers - developers, designers, writers, junior lawyers, accountants, analysts. Millions of people globally.

Phase 2 (2-3 years): Their spending disappears. They're not buying houses, cars, holidays, eating out, renovations. Hospitality, retail, trades all take a massive hit.

Phase 3: Those hospitality/retail workers lose jobs. Now you've got even less consumer spending. More businesses collapse.

Phase 4: Tax revenues collapse. Government can't fund services. Universities lose students (why get a degree for a job that won't exist?). More job losses.

And like you said - who's buying the AI services if nobody has money? OpenAI, Anthropic, etc. are banking on massive enterprise adoption, but if enterprises are shedding workers to cut costs, where's the revenue growth coming from?

The AI companies seem to think they're creating "productivity gains" but they're actually just... removing the customers. It's like automating away your own market.

And the baffling part: the 70-year-olds in the House of Lords aren't thinking about this because:

• They've got pensions and property
• They don't understand the speed of change
• They think "people will retrain" (into what?)
• They assume the market will sort itself out (it won't)

There's no social safety net designed for "30% of knowledge workers lose their jobs in 18 months." Universal Basic Income gets discussed but nobody's actually implementing it. Retraining programmes are a joke when the new jobs also get automated.

The terrifying bit: We might be watching the beginning of a genuine economic collapse and everyone's just... carrying on as normal? Like the Titanic where the band keeps playing?

What do you think actually happens? Mass unemployment and UBI? Social unrest? Or am I being apocalyptic and there's some economic adjustment I'm not seeing?

 

 

Exactly. UBI only works if there's a productive economy to tax. If you've automated away the workers, who's generating the wealth to redistribute?

The standard answer is "tax the AI companies and corporations" but:

• They'll offshore/restructure to avoid it
• If you tax them at 90%, they leave (Ireland, Singapore, wherever)
• Even if you catch them, the math doesn't work - there aren't enough mega-corps to fund 20 million people

And you're right about the spiral:

• Tax the remaining employed at 60-70% to fund UBI
• They emigrate or stop working (why bother?)
• Tax base shrinks further
• Benefits get cut
• Social unrest
• Economy collapses further

The billionaires/AI companies will say: "AI creates new jobs we can't imagine yet, just like the Industrial Revolution"
 

But that's bullshit because:

• Industrial Revolution took 100+ years to stabilize
• It created physical jobs (factories, infrastructure)
• AI is eliminating jobs faster than any historical parallel
• And it's coming for the new jobs too (AI training AI, AI managing AI)

What actually happens?

Probably some nightmare scenario like:

• Mass unemployment but no political will for real UBI
• Degraded benefits system, means-tested poverty
• Huge wealth concentration at the top
• Social unrest, possibly serious civil disorder
• Authoritarian responses to maintain order
• Economic depression that makes 2008 look mild

Or maybe I'm wrong and there's some magical economic rebalancing. But I don't see it.
 

The thing that keeps me up: Developers/designers are watching their own industry die in real-time, and they're still not sure what to do about it. If they can't figure it out with their skills and awareness, what chance does a 50-year-old retail manager or paralegal have?

Does any part of you think there's a way this doesn't end badly?

What actually happens?

Probably some nightmare scenario like:

• Mass unemployment but no political will for real UBI
• Degraded benefits system, means-tested poverty
• Huge wealth concentration at the top
• Social unrest, possibly serious civil disorder
• Authoritarian responses to maintain order
• Economic depression that makes 2008 look mild

 

This all sounds very conspiratorial but probably true!  We will own nothing but be happy.

[danny.]

14 minutes ago, pazzerfox said:

What actually happens?

Probably some nightmare scenario like:

• Mass unemployment but no political will for real UBI
• Degraded benefits system, means-tested poverty
• Huge wealth concentration at the top
• Social unrest, possibly serious civil disorder
• Authoritarian responses to maintain order
• Economic depression that makes 2008 look mild

 

This all sounds very conspiratorial but probably true!  We will own nothing but be happy.

Yup, can't see how else it plays out. We needed to have legislated 5 years ago. This will make climate change look mild in comparison.

Even if there was political will for UBI there would be no way to collect enough tax to implement it.

[Trumpet]

I don’t keep myself up to date well with AI but I’m pretty sure a lot of people underestimate it because of their experience with ChatGPT. I’d imagine it’s way ahead of what we get our hands on, particularly as a free to use user?

[leicsmac]

Potentially another source of increased inequality, that can lead to increased conflict, that can lead to catastrophe.

 

Not sure what the solution is on that one, but someone smart needs to come up with it and have the power to apply it. Same as other big issues of similar type. 

[Greg2607]

To give you an idea of the scale of all of this, I'm going to use a single job family.   Drivers.

 

Globally, there are around 200m people employed in a professional driving role. (taxi's, haulage etc) - look at what Tesla (RoboTaxi)and Waymo are doing and then extrapolate that out across the globe.  200 million people displaced from roles... MAINLY low skilled roles (no offence to any professional drivers) - And where do they go? - because lots of the other "low skilled" entry level jobs are being automated away. As soon as enterprises can do away with the human element of transport... it becomes a 24 hour service and hugely more productive.

 

Now imagine a world where every single customer service job disappears. First line conversations will be with AI Agents. Another huge volume of workers. 

 

The entire fabric of society is going to have to change. 

 

I imagine the vast majority of us can barely imagine what that future looks like, let alone prepare ourselves for it, or plot a path to remain in secure employment. 

 

Edited 29 January by Greg2607

[Grebfromgrebland]

Unless of course we reject AI taking our jobs and society away from us. 

 

There's over 60M people in this country we should set the agenda not Google, palantir or open ai. Once we lose control we never get it back. 

 

 

[Greg2607]

2 minutes ago, Grebfromgrebland said:

Unless of course we reject AI taking our jobs and society away from us. 

 

There's over 60M people in this country we should set the agenda not Google, palantir or open ai. Once we lose control we never get it back. 

 

 

sure... how do you propose to do that??

 

because in theory, once the human labour element is stripped out of services... by definition they become cheaper to provide than those that still require human labour. 

 

most people are going to purchase a service based on cost. 

 

 

if a taxi with a human driver is 3 times more expensive, are you going to pick that one? 

 

[Mike Oxlong]

If you strip away jobs you diminish the number of consumers so the only way to rectify the problem is by providing universal benefit raised by taxing the companies making larger profits as a result of having reduced their workforce and labour costs 

[danny.]

2 hours ago, Trumpet said:

I don’t keep myself up to date well with AI but I’m pretty sure a lot of people underestimate it because of their experience with ChatGPT. I’d imagine it’s way ahead of what we get our hands on, particularly as a free to use user?

Yes, if you just have the odd chat with free plan ChatGPT in a browser you're probably accessing 5% of where we are right now with AI. Look into Codex, Claude Code, Clawdbot etc plus lots of targeted apps that might use the same LLMs behind the scenes (for example Figma Make) and there are 10s of millions of jobs it can already replace.

[leicsmac]

There is definitely going to have to be a massive change in the working paradigm in a lot of first world nations because of AI.

 

But, that being said, again it's a problem of inequality of resources instead of actual, genuine scarcity of them, and is therefore at least theoretically solvable. 

[lcfc sheff]

Need to ban AI for anything other than healthcare (maybe some other things i’m

not thinking off)