A MESSAGE FROM OUR SPONSOR

How can healthcare organizations capture the next wave of AI value?

Many healthcare systems and hospitals have deployed AI to address specific clinical and operational challenges. But without enterprise-wide integration, these tools can create fragmentation and limit impact. Real transformation will require a more connected approach.

In The coming evolution of healthcare AI toward a modular architecture, McKinsey explores how shifting to a modular, interoperable AI architecture can unlock scale, improve data governance, and enable holistic innovation. The article outlines the strategic moves leaders must make now to build a more resilient, AI-enabled healthcare system.

Read the full article here

A few months back, OpenAI, the Musk antagonist and Trump administration ally, released some data that contained stirring and deeply unsettling insights into the fragility of the increasingly artificial human condition. In response to a wrongful-death lawsuit filed against the company, OpenAI announced in October that 0.15 percent of its weekly users have had “conversations that include explicit indicators of potential suicidal planning or intent.” Some 0.07 percent of users, meanwhile, showed signs of “serious mental health concerns, such as psychosis and mania.” Those numbers add up: That’s about 1.2 million and 560,000 people, respectively, each week, based on ChatGPT’s contemporaneous base of 800 million weekly active users.

Like it or not, A.I. has become inextricable from mental healthcare. According to reports published in 2024 and 2025 by Harvard Business Review, people use generative A.I. for therapy and companionship more than anything else. (A more recent HBR report on Gen Z, interestingly enough, found companionship to be a significantly less popular use case.) Nevertheless, there are enough users to form a subreddit of 23,000 people called TherapyGPT—a community full of stories of users going so far as to abandon their human therapists entirely in favor of artificial intervention.

The appeal isn’t hard to divine. An L.L.M. is always there: cheap, readily available, nonjudgmental, and it doesn’t require insurance. And yet, obviously, therapy bots are neither trained nor licensed; they remain unregulated and, consequently, lacking in the kind of rigorous evidence that would suggest they’re either effective or safe for therapy. Last year, the American Psychological Association called the use of generic chatbots for mental health support a “dangerous trend.” APA C.E.O. Arthur Evans said at the time that “APA envisions a future where A.I. tools play a meaningful role in addressing the nation’s mental health crisis, but these tools must be grounded in psychological science.”

It’s an effort that Limbic, a British company operating at the frontier of mental health A.I., has been undertaking since its foundation. And in a paper published Thursday morning in Nature Medicine, Limbic provided the results of a double-blind, randomized controlled trial that found, among other things, that A.I. actually outperformed human therapists at therapy.

Caveats, of course, abound. The study was specific to text-based “psychotherapy interactions”; the outperformance was across “key clinical competencies required for delivering high-quality cognitive behavioral therapy,” rather than therapy sessions and formats writ large; the study focused on single sessions rather than multiple sessions; and participants were told that they were evaluating therapy agents—rather than receiving a full-on mental healthcare intervention—“and were asked to construct a realistic scenario to discuss.” Still, a consortium of 22 clinicians blindly reviewed transcripts resulting from both A.I. and human sessions (the study featured only six human therapists as a comparator), and the result was that “74.3 percent of all cognitive layer sessions surpassed the top 10 percent of scores given to the human therapists.”

A MESSAGE FROM OUR SPONSOR

How can healthcare organizations capture the next wave of AI value?

Many healthcare systems and hospitals have deployed AI to address specific clinical and operational challenges. But without enterprise-wide integration, these tools can create fragmentation and limit impact. Real transformation will require a more connected approach.

In The coming evolution of healthcare AI toward a modular architecture, McKinsey explores how shifting to a modular, interoperable AI architecture can unlock scale, improve data governance, and enable holistic innovation. The article outlines the strategic moves leaders must make now to build a more resilient, AI-enabled healthcare system.

Read the full article here

Limbic was intent on clinically validating one of its two core products: a clinical reasoning harness designed to make standard L.L.M.s safe and effective for therapeutic interventions. This model-agnostic architecture leverages custom-built machine learning models trained on clinical mental health information to closely control, guide, and review a given L.L.M.’s responses. Today, Limbic works directly with clinicians; one of its offerings involves a (similarly validated) A.I. system to aid in patient intake and assessment. The other, called Limbic Care, is an A.I. system designed to integrate into existing care plans.

That product, which has already been deployed, was the subject of the study, which also found that the A.I. agents—when imbued with Limbic’s specialized clinical layer—“significantly outperformed stand-alone L.L.M.s.” The clinicians preferred those specialized agents over the stand-alone L.L.M.s 82 percent of the time. The findings, according to the paper, “provide early empirical evidence that L.L.M.s enhanced by specialist A.I. systems show capabilities required for interpersonal psychotherapeutic interactions.” In a real-world observational analysis of 9,000 users across the U.S. and U.K., the study found that users with higher exposure to the clinical layer reported a recovery rate 20 percentage points higher than those with lower exposure.

Professor Tobias Hauser, a computational psychiatrist and senior author of the study, offered various caveats, but noted that the big takeaway was that “standard models are not made for mental health support and are much more likely to fail. To provide good mental well-being support, one needs to divide the excellent language abilities of the L.L.M. from the reasoning and clinical expertise that is done by other, complementary models. Simply put, someone who can talk well and fluently might not be the best at providing therapy.”

Dr. Ross Harper, Limbic’s C.E.O., told me that the company decided from the beginning not to attempt to solve the language modeling problem. Instead, he was focused on addressing the “thorny question of how you can unlock this technology within clinical settings in a way that is not only clinically validated and performant, but also explainable and auditable.” The results, he said, give Limbic and the field a “clear path to how we can use this new technology within a mental healthcare setting.” A.I., he added, will be adopted in healthcare “at the speed of trust.”

Limbic remains focused on working with clinicians, but Harper didn’t rule out pursuing a direct-to-consumer offering eventually. Even then, though, he doesn’t expect therapists to go away. “There is an intractable supply-demand imbalance in psychological therapies. We have graduated existing human clinicians into more senior roles, where they will spend every minute of their valuable time working on the things that only they can do, like handling complexity,” he told me. “It’s about deploying our scarce resources in a way that actually scales the industry. That’s how I see things playing out in practice. I think it’s sensational to talk about substitution or replacement.” He added that the company, which was backed by Khosla Ventures in a $14 million round in 2024, experienced an “inflection point” in growth last year, though he declined to elaborate.

Perhaps none of this is perfect or all that shocking: Humans require support, technology has always been a means through which people express their id, and therapy is one of many fields where A.I.’s great potential is still mostly unrealized, and far from straightforward. “I don’t think this is our golden ticket to resolving our mental health crisis or the shortage of therapists out there,” Dr. Zachary Ginder, a California-licensed psychologist and a practicing therapist, told me. “But that said, it’s promising.” Dr. Sahib Khalsa, a psychiatrist and neuroscientist at UCLA, agreed, telling me that the study marks “a step in the right direction. I’m optimistic about the possibilities, but I’m also skeptical until there’s a body of evidence.”

Ginder added that he could see a lot of potential value in something like Limbic—say, where it acts as a bridge between sessions, an option while people are stuck on a waitlist, or even a decent, frictionless option for “subclinical” people. “As far as I’m concerned, if we’re looking at it as support for therapists, then it’s not like one has to be better than the other.” But as a stand-alone option, he said, the risks are significantly higher.

As machines become both more normalized and more complementary to human specialists, in this field as in others, Ginder noted that some of the most impactful aspects of therapy remain rooted in a kind of humanity that the machines of today, at least, can’t really automate. “We have a thing called co-regulation in therapy: voice pacing, a person’s presence, just attunement with another person,” Ginder said. “And we are biologically hardwired for interaction and relationship. I often find that healing happens in relationship—so that’s missing from this process. I think that leads to the conclusion that this won’t be for everyone, but for a certain population I think it could be very beneficial.”

As Congress remains stalled out on frameworks and attempts to actually regulate A.I., China continues to do the opposite. And increasingly, the Chinese government has turned its legislative attention to A.I. companions. [Carnegie Endowment]

This piece offers a good look at the realities of A.I. delusion and the many different forms it takes. [404 Media]

Somewhat relatedly, in a recent study undertaken by CNN and the Center for Countering Digital Hate, chatbots helped teens (hypothetically) plan shootings. [CNN]

Grammarly has already been served with a class-action lawsuit over its recent rollout (since rolled back) of an A.I. tool that claimed to offer writing insights from real-life authors, journalists, and academics. In keeping with the tradition of the A.I. industry, they didn’t ask for consent first. [Wired]

That’s all for today. I’ll see you next week.
Ian

Need help? Review our FAQ page or contact us for assistance. For brand partnerships, email ads@puck.news.

You received this email because you signed up to receive emails from Puck, or as part of your Puck account associated with {{customer.email}}. To stop receiving this newsletter and/or manage all your email preferences, click here.

Puck is published by Heat Media LLC. 107 Greenwich St., New York, NY 10006