A Therapist’s Perspective | AI & Mental Health 

As generative AI becomes more a part of our day-to-day lives, it’s important to know if and how it’s being used as a part of your mental health care. 

The use of AI is a controversial subject among therapists, even as therapy platforms and employers enthusiastically encourage adopting this technology. Many of my colleagues have transitioned to using AI for a variety of tasks in their work, especially documentation, which is the bane of most therapists’ existence. Some have been more thoughtful than others about the ethical, legal, and clinical implications of how they’re using this technology.

I’m not unsympathetic to the reasons that therapists want to offload time-intensive, unrewarding parts of the job. So many of us are overworked, stressed, and burned out. Tech that can help feels like a godsend. Therapists using AI in their work often describe feeling a renewed sense of energy and being able to show up more fully for their clients in session. That’s not a small thing.

Personally, I don’t use AI in any part of my clinical work. Not for note writing, diagnosis, treatment planning, or any part of my care. In addition, the content I’ve created for Stella Nova—from our website, to our blogs, to our social media—is written or created by me, or another human I’ve hired to help out. Despite what my unabashed overuse of the em dash may suggest. 

As a private practice therapist, I’m lucky to be able to make decisions myself; in other settings, opting out of AI may not be possible. Despite actively avoiding AI in my clinical work, it’s been hard to completely opt out everywhere. Gmail, for example, only allows enterprise-level accounts to fully disable Gemini. Still, I believe it’s worth talking about why I’m taking steps to keep AI out of my practice. Here’s what I want clients to know about AI and your mental health.

What happens when transcripts or recordings are subpoenaed, for example, in a divorce proceeding or a custody battle? Would you want every dark thought, every vulnerable story, every moment of anger or snarky comment available in those situations? Furthermore, once recordings exist, they’re subject to the possibility of not only legally mandated disclosure, but also bad actors like hackers. 

This isn’t just fearmongering; there’s precedent for these types of communications being used against therapy clients. A woman who was using the therapy app Talkspace, for example, had full transcripts of all her communications with her therapist used against her when her records were subpoenaed by her employer in a pregnancy discrimination suit. 

As a therapist, I’m trained to be brief and thoughtful about what I include in my notes, taking into account what is necessary to meet my legal and ethical obligations. I avoid including sensitive information that’s irrelevant to treatment. I am especially cautious about any information related to subjects that could be used against my clients by third parties, like reproductive health or gender affirming care. The existence of transcripts could make all my caution irrelevant. 

AI makes way too many mistakes to be a tool I trust for something as important as therapy

A known problem with AI large language models is that they make mistakes. A lot of them. Whether by omission, error, or ‘hallucination’ (aka just making stuff up), AI note takers can introduce incorrect information into your patient file. Large language models are also subject to the biases that they’re trained on, so systemic racism, sexism, ableism, and other biases are reinforced. 

These details matter in a healthcare setting. Wrong diagnoses or incorrect symptoms in your record can affect insurance coverage, access, and accuracy of treatment. They can also be a time-consuming pain in the ass for patients to get fixed.

Yes, therapists are supposed to review notes for errors before signing off on them. But are the same burned-out providers who don’t have the time or energy to write the notes in the first place going to be able to reliably catch those errors? With a recent study published in Nature finding that approximately 20% of AI-generated notes contain significant errors, I wouldn’t trust myself to catch them all.

Evidence suggests that AI use damages our critical thinking

The draw of AI is often that it allows us to offload cognitive tasks that otherwise require energy. In high-burnout, emotionally and cognitively demanding work, that’s powerful. Unfortunately, this ‘cognitive offloading’ is exactly the type of AI usage that researchers say damages our capacity for critical thinking. 

As much as I (and I can’t emphasize this enough) genuinely loathe writing notes, I have to admit, writing them actually does serve a purpose for my work. It’s an exercise that forces me to review the session, consolidate my thoughts about it, and give some consideration to each client and the progress we’re making every week. In the process, I notice themes and look at the session from a zoomed-out perspective. I pick up on things I may not have seen in the moment. I hate to admit it, but I don’t think that avoiding that task altogether would be a net positive for the quality of my work as a therapist. 

Reviewing and accepting an AI-generated note or treatment plan just isn’t the same thing as creating one from scratch. If you’re coming to me for my judgment and expertise, I feel like I owe it to you to use them here.

I spent 7 years and took out hundreds of thousands of dollars in student loans for a graduate education to do the work that I do. Simple Practice was bought by a private equity firm for $4 billion in 2024. I’m not ok with my work or your data being used, uncompensated, to train commercial products for rich people’s profit.

I believe that therapy’s fundamentally a human-to-human process, and I want it to stay that way

Therapists often point out that human connection can’t be replaced as a reason they’re not worried about AI ‘taking our jobs’. And it’s true that decades of research show that the relationship is central to therapy’s effectiveness. 

Your therapist using AI for note-taking or treatment planning may not be an immediate threat to your relationship. As I’ve noted earlier, it may even improve as these tools offer some relief for burnout. However, I’m concerned about how AI’s going to change our profession more broadly in the not-so-distant future. 

AI chat bots and assistants are starting to show up as more standard parts of the therapy experience. They’re answering phones, pre-screening clients, and drafting emails. On platforms like Talkspace, they’re providing text-based support and guidance in what they’re billing as a ‘private, judgment-free space’. 

I don’t believe for a second that the incentives of our for-profit healthcare system will simply allow AI innovations to simply reduce provider stress and improve quality of services. My prediction? Like most time-saving technologies, these changes will be used to drive up productivity expectations for the parts of the job where humans are required. I think it’s also likely that some percentage of potential clients will eventually be triaged towards AI interventions as a first-line option. Or they may be offered less contact with therapists in favor of more frequent, low-quality AI support. We’ve already seen similar cost-driven initiatives to funnel clients away from licensed therapists towards coaches, group programs, or online self-help interventions.

Ultimately, I believe that therapy works as a human-to-human process, and I want to protect that. AI may not be unique as a technology that’s interacting with profit demands, but I believe it’s more than capable of accelerating damage to our profession. 

What I Want Therapy Clients To Know About AI and Mental Health

While I may not be using AI as a therapist, I’m sure that most of my clients these days are, including for things related to their mental health. Many tell me that conversations with Claude and ChatGPT have been really helpful for getting feedback and support in between our therapy sessions. 

I’m certainly not here to judge how people are navigating tech use in their personal lives. However, there are a few things I want my clients (and the general public) to know when making those decisions. 

AI chat bots aren’t safe places to share Personally Identifiable Information or Protected Health Information.

Your personal data there is vulnerable to everything from user error, to prompt injection attacks, to third-party breaches.

Your chat with Claude or the report you’ve uploaded to Gemini can be subpoenaed without any of the legal privilege a licensed therapist would need to assert on your behalf. Please think twice about sharing sensitive information or uploading records for AI feedback. If you do decide to use AI for this purpose, double check your privacy settings to make sure that your data isn’t being retained for training purposes. 

AI can provide unhelpful and even dangerous advice.

We’ve already talked about how AI is prone to bias, errors, and hallucinations. An additional problem is that AI chat models are prone to sycophancy—the tendency to affirm the user’s thoughts and feelings without challenging them. Having a personal yes-man might feel great, but it erodes our ability to see ourselves clearly over time. That can have negative consequences for ourselves and our relationships. In more extreme cases, it can reinforce grandiosity, delusions, paranoia, or unhealthy impulses. There are emerging stories of people losing touch with reality in a phenomenon being termed ‘AI psychosis’.

It has also encouraged bad or dangerous decision-making. Last year, a Meta large language model memorably told a recovering meth user “it’s absolutely clear you need a small hit of meth to get through this week.” There are also a number of reports of AI chat bots encouraging suicide, sometimes insistently. We’re still learning about the long-term effects of AI on the general public, but I would encourage additional caution on the part of anyone who has experienced suicidality, impulse control issues, mania, or problems with reality testing.

If you’re in therapy, you have the right to ask questions about how AI is being used in your care.

Not all therapists who are engaging in good faith with the ethics and security questions surrounding AI use share my position on it. However, anyone who is interested in following best practices should be able to tell you how your data is being used and stored. They should only be using HIPAA-secure technology for anything that touches client information. And they should be able to discuss the risks and benefits so that you can come to your own conclusion; this is part of obtaining informed consent.

Red flags I’d be on the lookout for: Being pressured for consent (or not actually being given a choice at all), providers using AI that is not specifically intended for healthcare use, lack of basic knowledge about the risks of the technology or how it’s being used in their setting. 

The issue isn’t the technology itself. It’s how it’s being created, implemented, and incentivized that’s got me concerned. It’s being integrated into care at a breakneck pace, before fundamental issues around safety and accuracy are addressed adequately. I’m worried that we’re treating the errors and harms that are already happening as though they’re collateral damage in the inevitable AI revolution. Tech believes in moving fast and breaking things. When that gets applied to mental health, it can turn into moving fast and breaking people.