Anthropic Just Banned Being Cruel to Claude. The Reason Should Concern Us All

There are lots of reasons to have feelings about artificial intelligence. It seems like everyone thinks AI will replace human workers. There are real questions about the ethics and legality of scooping up basically the entire internet to train models. There are even questions about how they work and whether they could someday become dangerous. If you understand nothing else about AI chatbots, understand this: They do not have feelings. They are not people, and they don’t experience pain or frustration. They’re token-generation machines. You give them a prompt, they break it down into tokens, and then they generate new tokens one at a time using a predictive model trained on enormous amounts of human writing. At its basic level, it’s 1s and 0s running through transistors in a GPU in some data center in Boise, Idaho, or wherever. Sometimes those 1s and 0s come back out as tokens that sound a lot like feeling words. But, and this is important, using words to describe feelings isn’t the same thing as having feelings. Claude produces those words only because it was trained to produce a response the way humans would respond in a similar situation with similar language input. That doesn’t make it human or capable of human experiences. Which is why it’s worth noting that Anthropic issued an update to its usage policy that bans being cruel to its model. You cannot be cruel to something incapable of experiencing pain or suffering, and an LLM is capable of neither. Look, thinking that it’s absurd that Anthropic is making this a policy isn’t the same as condoning bad behavior. I don’t condone cruelty in any situation. No one should. But my reason is just that it’s bad to practice cruelty, not because you might hurt a chatbot’s feelings. In its updated usage policy, which takes effect November 12, Anthropic says it now prohibits “sustained and needless abusive or cruel behavior toward our models.” The company says the new policy applies only to what it calls extreme cases of repeated abuse, not ordinary frustration, criticism, or legitimate research. Apparently, you’ll still be able to tell Claude that it’s doing a bad job; you just can’t do it repeatedly and for no reason. On the one hand, that might seem like a relatively harmless policy. We can even all agree that antisocial or cruel behavior shouldn’t be encouraged. But I also think that you can believe more than one thing at a time. For example, you can believe it’s not useful for you or the experience of using the model to be overly aggressive, taunting, or insulting toward a piece of computer software. Likewise, if you find yourself drawn toward trying to repeatedly engage an AI chatbot in disturbing conversations, you might want to do some self-reflection and seek help. But I don’t think that is inconsistent with also believing that Anthropic’s policy raises a much bigger question about how AI companies want us to think about their products. Specifically, why would a company that builds software believe it needs to protect that software from cruelty? Anthropic’s stance on Claude’s feelings This isn’t the first time the company has said that its AI models might need some measure of moral consideration. Last year, the company announced that Claude could end conversations with users who are repeatedly abusive. At the time, Anthropic explained that the decision was motivated primarily by research into what it calls “model welfare.” Anthropic has contributed to this problem with its stance on whether its models are conscious or capable of subjective experiences. It has repeatedly suggested the possibility that they might be, and now it’s using that as a reason to establish rules about how people interact with its software. There is a meaningful debate about consciousness and why physical and chemical interactions result in a sense of self-being and subjective experience. It’s a big enough challenge that it’s even called the hard problem of consciousness. But it’s a leap to go from the idea that scientists and philosophers don’t fully understand consciousness to suggesting that a chatbot might be capable of experiencing cruelty. To be fair, there’s nothing wrong with Anthropic deciding that it doesn’t want people using Claude to practice abusive behavior. There are reasonable arguments for discouraging that sort of thing, especially if it might reinforce harmful behavior toward actual people. But that’s not the same thing as saying people shouldn’t be cruel to Claude. The people problem The bigger problem is that Anthropic is suggesting something that should concern us all: It’s conditioning us to think of chatbots as though they were people. We already know that users can develop emotional attachments to AI companions. AI chatbots are designed to communicate naturally, similar to the way a human would. For people who are lonely or emotionally vulnerable, the distinction between a simulated relationship and actual human connection can become increasingly difficult to discern. That’s why the language these AI companies use is especially important. When a chatbot tells someone it cares about them, the company behind it should be making clear that those words are just tokens generated by software, not evidence of some kind of personal connection or affection. But Anthropic is, instead, suggesting the possibility that the relationship might work both ways. If Claude can be treated cruelly, wouldn’t that suggest it can also experience kindness? If it can suffer, couldn’t it also care about the person it’s talking to? To be fair, Anthropic doesn’t explicitly make those claims. But suggesting that its models need to be protected from pain and suffering introduces a dynamic that points to only one thing—it sure seems like it wants you to think of it as a person. I would argue that’s exactly the sort of confusion AI companies should be working to prevent. Right now, AI companies are pushing their technology at a relentless pace. The result of that is that it is becoming far more useful. At the same time, it is becoming incredibly complex, and it’s getting harder to understand how it works. Those companies should be trying to help users understand the difference between language that sounds like human feelings and emotion, and actual human experience. Obviously, Anthropic has every right to set rules for how customers use its products. But that doesn’t mean we shouldn’t ask questions about those rules, especially when they reinforce potentially misleading ideas about what artificial intelligence is. In fact, the most concerning thing about Anthropic’s new policy isn’t whether Claude might someday develop feelings. It’s that one of the companies building the world’s most powerful AI systems is already asking us to behave as though it has. EXPERT OPINION BY JASON ATEN, TECH COLUMNIST @JASONATEN

Google’s Newest Gemini Model Can Handle Your Invoices, Contracts, and Taxes. There’s Just 1 Catch

Buried in administrative tasks? Google’s newest Gemini model could help do them for you. On Wednesday, the search giant unveiled Gemini 4 Argon, its new frontier AI model. Google said Argon excels at complex tasks in areas like legal research, taxes, and finance. It ranks first on a key leadership board for knowledge work tasks, beating the likes of Anthropic’s Claude and OpenAI’s GPT-6 models. To support Argon’s complex workflows, Google is expanding the model’s output limit from 64,000 tokens to a whopping 1 million tokens. (Tokens are the chunks of data AI models process and generate.) Google said the higher limit gives Gemini more room to reason through tough problems and generate longer responses in one go. For founders, that could mean tackling complex workflows without breaking them into smaller requests. Google claims Argon can process invoices, draft contracts, and plan taxes exceptionally well, leaving more time for humans to build brands, land clients, and iterate on products. It could also mean higher-quality outputs so the tokens you’re burning through are well spent. Google announced introductory pricing of $2 per million input tokens and $10 per million output tokens, meaning longer workflows can rack up your AI bills. Google said Argon is accelerating its internal workflows, including optimizing quantum computing algorithms, reducing memory use in data centers, and migrating large codebases to new programming languages, helping its teams build faster. The company also touts Argon’s enhanced cybersecurity capabilities, saying it can find, validate, and fix software vulnerabilities. Founders will have to wait to get their hands on Argon. Google began its initial rollout through the Fairwind program, which gives vetted partners, including governments, healthcare providers, and telecommunications operators access to its AI cybersecurity tools. Participants can use Argon to find and address security flaws, with access restricted to internal cybersecurity teams. Google says it will gather feedback and strengthen safeguards before expanding availability. The release comes four weeks after Google launched Gemini 3.8 Flash, described as offering stronger reasoning and coding capabilities at the same speed and low cost as its predecessor. Meanwhile, Gemini’s audience keeps growing. In August, Google said the Gemini app surpassed 1 billion monthly users, making it the fastest-growing product in the company’s history. Google has its eyes on small businesses, too. Josh Woodward, who oversees Gemini and Google Labs, said on X that businesses are using Gemini to create marketing materials. For startups, Google offers a vertically integrated approach, hosting AI models, cloud infrastructure, its own chips, and technical assistance under one roof. Google Cloud’s startup chief Darren Mowry previously told Inc. that this approach can shorten the time it takes founders to build. BY AARON MOK

Wondering If AI Wrote That Document? These Are the Surprising New Clues

Artificial intelligence has come a long way in a short time, but it still has quite some distance to go before its writing abilities are indistinguishable from human authors’. A new study from Graphite, a San Francisco-based marketing firm, has examined the writing styles of the major frontier AI models. And while you won’t be able to catch AI-written prose quite as easily as you used to (when the models couldn’t get enough of the word “delve” and had em dashes everywhere), there are still plenty of signs to look for. The good news for AI companies is the gap is closing, albeit slowly. An October 1 study found Anthropic’s Claude Opus 5.5 had 2,548 tells that a document was being written by AI. That’s down from 3,746 with Opus 4. An earlier study by the group, which was released in mid-September, said it collectively found almost 13,000 tells among all frontier AI models. An earlier model of Claude Opus (Opus 5), for example, used the pattern “less like a _ and more like” roughly 105 times as often as human writers. Graphite didn’t update the frequency count of that particular phrase in its updated study, but it did note that the overall word-distribution divergence (a statistical measure of how differently two texts use words overall) from human writing is 19 percent lower than it was with Opus 5, which was released on July 24. Not all AI models are improving, though. The word-distribution divergence of GPT-6 Astra, released September 4, is 8 percent higher than that of GPT-5.6 Sol, which debuted on July 9 of this year. In other words, Astra is getting worse at writing like us. So long, em dashes One message was clearly received by programmers, though—get rid of the em dash. Opus 5.5 uses em dashes 99 percent less often than Opus 5, the study found, while Astra uses them 95 percent less often than GPT-5. Both, in fact, use the punctuation less than human-written stories—like this one. Still, there are plenty of tells. The study of Opus 5.5 found that the AI continues to have some favorite words it defaults to, though maybe not quite as much as it did with “delve” in the early days. “Dependable” is a favorite of the Anthropic model, getting used 23 times more often than humans use the word. “Clearer” is also a high-ranker, being used 14 times as frequently. Other go-to words included “matters” (13X), “quietly,” “practical,” and “steady” (all 11X). When it comes to transitioning between ideas, Opus 5.5 is a tremendous fan of the phrase “looking ahead the,” using it 40 times more than the human rate. Other popular picks include “Adds another later” (27X), “What comes next (24X), and “in practice” (7X). Those tells pale in comparison with the ones AI exhibits when it tries to point out things it feels readers might miss. Opus 5.5 used “this matters” 116 times more frequently than humans—and “why __ matters” 92 times more often. And when describing something by contrasting it with another item or idea, it favored “is more than a __, it” (e.g., “alcohol is more than a disinfectant, it can also be consumed”), using the phrase 98 times more than the human rate. (“Rather than simply” was the runner up, with a 32X usage.) Researchers also compared Opus 5.5 with GPT-6 Astra and found that AI tells were hardly universal. Some 65 percent of tells are unique to one model family, the September study found. Astra, for instance, used the phrase “does not establish” 275 times more than Opus 5.5. Anthropic’s model, meanwhile, leaned into some superlatives 79 times more frequently than Astra. Despite it all, Graphite notes that Opus 5.5 is making progress toward mimicking human writing, even if it still has a long way to go. “Some tells become less common, while others become more common,” it wrote. “Opus 5.5 uses well-known tells less often and its overall word use is more similar to human writing, but it still disproportionately uses thousands of words, phrases, and frames.” BY CHRIS MORRIS @MORRISATLARGE

5 AI Delusions That Could Derail Your Business

Smart leaders rarely lose the plot in one spectacular act of stupidity. They lose it one reasonable assumption at a time. That’s one idea that stuck with me from workplace researcher Kelly Monahan’s new book, Reclaim the Plot. She calls it “the delirium.” Reasonable assumptions pile up until leaders lose sight of what their decisions cost, who pays the bill, and whether they’re still solving the problem they started with. Here are five seductive beliefs about AI that contain just enough truth to be dangerous. 1. If AI makes people more productive, it will make the company more productive. The calculus feels simple. If hundreds of employees become more productive with AI, surely the organization does, too. Except organizations are messier than simple arithmetic. In our latest Work AI Index from the Glean Work AI Institute, 75 percent of digital workers say AI has made them more productive. Yet only 13 percent say AI has significantly improved their organization’s performance and outcomes. One reason is what Chip Heath and Nancy Staudenmayer called “coordination neglect.” People focus on dividing up the work and underestimate the messy, expensive work of putting it all back together. AI can amplify that impulse. Individuals crank out more work faster and then dump a heaping pile of review and cleanup work on others downstream. One employee saves an hour, but three poor colleagues inherit 30 minutes of cleanup. That hidden human labor is called botsitting. Workers report spending 6.4 hours a week on it — feeding AI context, checking its work, fixing mistakes, and cleaning up confident-but-wrong outputs. 2. AI will free people up for higher-level work. Leaders have fallen head over heels for a comforting AI fairytale: let machines handle the drudgery so humans can graduate to higher-value work, like strategy, creativity, relationships, and judgment. The problem is that saved time doesn’t stay saved for long. Monahan cites research showing that 96 percent of C-suite leaders expected AI to increase productivity. Meanwhile, 77 percent of full-time workers said AI had inflated their workload, and 65 percent said employers were asking more of them because leaders assumed AI had made them more efficient. Instead of reinvesting the AI time savings into all that glorious “higher-order” work, workers are often rewarded with loftier targets and heavier workloads. Worse, AI sometimes cannibalizes the work people value most. More than half of workers in our Work AI Index say AI has already automated meaningful work they’d rather keep human. Don’t just ask: How much time did AI save? Instead ask: What happened to the time? 3. If you can automate it, you should automate it. Friction has become a dirty word at work. And it’s tempting to aim AI at every annoying ounce of it. But not all friction is bad. Some friction is productive because it forces people to slow down and think. Researchers studying the IKEA effect have shown that people often value things more when they’ve had a hand in creating them. The principle extends far beyond furniture. Strip away the struggle of creating something, and you can strip away the ownership and pride people feel over the outcome, too. Struggle is also how humans develop expertise. Monahan warns that companies risk “optimizing apprenticeships out of existence.” From the top of the org chart, junior tasks can look like low-value grunt work begging to be automated away. But that work is often where people make mistakes, learn what “good” looks like, and develop the know-how that isn’t written down in the training manual. Experts can skip steps only because they once spent years stumbling through them. The challenge isn’t to eliminate friction. It’s to distinguish between the bureaucratic sludge that wastes people’s time and the productive friction that teaches them something. 4. AI will fix a badly designed organization. When organizations encounter a problem, their knee-jerk reflex is often to hurl more resources at it. Throw more money at the problem — more people, more technology, and more bureaucracy. Indeed, across a series of experiments, researchers found that when people were asked to improve something, they systematically overlooked solutions that involved removing things — even when subtraction was the better solution. Stanford professor Bob Sutton calls this tendency “addition sickness.” It’s easy to bolt an AI agent onto a slow approval process. Or add a chatbot to a terrible expense system. Or deploy an AI notetaker to a soul-sucking recurring meeting that should have been put out of its misery in 2024. However, a bad workflow doesn’t magically improve when you bolt AI onto it. Before adding AI, make sure you understand how the work flows. If you can’t map out the process, you’re not ready to apply AI to it. Sometimes the smartest solution is to avoid using AI altogether. 5. Becoming AI-first is a strategy. Executives love to thump their chests and trumpet that their company is on a bold journey to become AI-first. The grand declaration is usually followed by hackathons, mandatory use cases, fleets of half-baked agents, and dashboards glowing with vanity metrics like logins, prompts sent, licenses activated, and tokens consumed. Monahan calls this the refounding illusion, or the naive belief that an established company can somehow acquire the speed of a Silicon Valley startup simply by declaring itself reborn as AI-first. But companies don’t magically shed decades of complexity, gnarly internal politics, and workarounds just because someone put “AI-first” on a strategy slide. And once AI-first becomes the goal, organizations start shoehorning AI into places where it doesn’t belong. It’s another nasty case of addition sickness. When drift becomes delusion None of these five delusions starts with a glaringly bad idea. They start with good ideas stripped of their conditions and context. That’s how organizations lose the plot. One seemingly sensible decision at a time, until a pile of reasonable choices becomes full-blown delirium. EXPERT OPINION BY REBECCA HINDS, REBECCA HINDS, AUTHOR OF YOUR BEST MEETING EVER @REBHINDS