Saturday, May 24, 2025
I Tested 5 AI Assistants—and What I Found Was Surprising
Recently, the Washington Post invited me to join a blue-ribbon panel of communication experts for an AI writing experiment. Tech reporter Geoffrey Fowler pitched the idea as an old-fashioned bake-off with a modern twist. He asked us to test five popular AI tools on how well they could write five kinds of difficult work and personal emails.
Why emails?
“It’s one of the first truly useful things AI can do in your life,” says Fowler. “And the skills AI demonstrates in drafting emails also apply to other kinds of writing tasks.”
In total, the panel of judges evaluated 150 emails. While one AI tool was the clear winner, the experiment highlighted the benefits of AI writing and communication assistants—and one big limitation.
Since we were asked to read all the emails blind, we did not know which were written by ChatGPT, Microsoft Copilot, Google Gemini, DeepSeek, or Anthropic’s Claude. Fowler also had us score emails he had written to see if we could distinguish between AI and a human writer.
The best AI writing assistant
The clear winner was Claude.
“On average, Claude’s emails felt more human than the others,” Fowler noted. Another judge, Erica Dhawan, said, “Claude uses precise, respectful language without being overly corporate or impersonal.”
DeepSeek came in second place, followed by Gemini, ChatGPT and, in last place, Copilot. Although Copilot is widely available in Windows, Word, and Outlook, the judges agreed that its emails sounded too much like AI. “Copilot began messages with some variation of the super-generic ‘hope you’re well’ on three of our five tests,” said Fowler.
While Claude won this competition, I later learned that my scores showed a preference for the human-written emails. And that’s because all the AI assistants had one big limitation.
According to Fowler, “Our five judges didn’t always agree on which emails were the best. But they homed in on a core issue you should be aware of while using AI: authenticity. Even if an AI was technically ‘polite’ in its writing, it could still come across as insincere to humans.”
My takeaway:
AI tools are great for outline, flow, and clarity of argument. But they’re often stilted, formal, robotic, and lack personalization, emotion, and empathy.
AI assistants have trouble with creativity because the architecture on which they’re based (large language models) generates content with “syntactic coherence,” an academic term for stringing sentences together that flow naturally and follow grammar rules. But as you know, rules are meant to be broken.
Steve Jobs broke the rules
For example, in 1997 Apple’s Steve Jobs launched one of the most iconic campaigns in marketing history. The company was close to bankruptcy, and needed something to attract attention and stand out.
Apple’s now-famous television ad—nicknamed “the crazy ones”—featured black and white portraits of rebels and visionaries such as Bob Dylan, John Lennon, Martin Luther King Jr, and others. The marketing campaign is credited with redefining Apple’s brand identity helping to save it from financial ruin.
If the writing had been turned over to AI, it wouldn’t have happened.
How do I know? Claude told me.
“If asked to create a slogan like Apple’s famous campaign in my default mode, I would almost certainly have written ‘Think Differently’ rather than ‘Think Different,'” Claude acknowledges. “My training emphasizes grammatical correctness. The proper adverbial form to modify the verb ‘think’ would be ‘differently,’ and I’d be inclined to follow this established rule.”
Claude says it can analyze why the campaign worked “after the fact … but generating that kind of deliberate grammatical rebellion doesn’t come naturally to me.”
AI doesn’t have a rebellious streak because—breaking news—it’s not human. Some bots might perform better than others at simulating human qualities in their writing samples, but they don’t have the one thing you have: a unique voice built on years of personal experiences and creative insights.
AI is a helper, an assistant. Use it to brainstorm ideas, clarify thoughts, summarize documents, and gather and organize information. Those are all important and time-consuming tasks. But while AI can enhance communication, it shouldn’t replace the communicator.
As more people rely on AI assistants to write emails, resumes, memos, and presentations, there’s a real danger that many people will sound alike—corporate recruiters are already spotting this trend.
But you’re not like everyone else. You have a unique and powerful story to share. Don’t let artificial voices silence your authentic one.
EXPERT OPINION BY CARMINE GALLO, HARVARD INSTRUCTOR, KEYNOTE SPEAKER, AUTHOR, ‘THE BEZOS BLUEPRINT’ @CARMINEGALLO
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment