Instagram’s AI Support Bot Made a Costly Mistake. It’s a Warning for Every Company

Sometime last weekend, hackers asked Meta’s AI support bot for access to someone else’s Instagram account. The bot said yes. The method, reported first by 404 Media, was almost insultingly straightforward: instruct the chatbot to add a new contact email to a victim’s account, confirm the change with a code sent to that email, then use it to reset the password. In some cases, all that was also needed was a VPN to spoof the victim’s location. Among the accounts taken over: the former White House page from Barack Obamaʼs presidency, cosmetic chain Sephora, and the chief master sergeant of the U.S. Space Force. While this hack was obviously a security flaw, most businesses should consider that this blunder also reveals a management error. The front door is now automated AI customer service is everywhere. According to MarketsandMarkets, the global AI customer service market was valued at $12.06 billion in 2024 and is projected to reach $47.82 billion by 2030, growing at a compound annual growth rate of around 25.8 percent. The economics are hard to argue with: AI scales cheaply, doesn’t require shift management, and doesn’t call in sick. AI has been shown to cut response times by 37 percent and reduce operational costs by 35 percent. Companies have moved quickly. Banks use it for dispute resolution, airlines use it for rebooking, telecoms route complaints through it, and e-commerce platforms use it to handle returns. Around 80 percent of companies are either currently using or planning to adopt AI-powered chatbots for customer service by 2025. Meta is not unusual for deploying AI support at scale. When Meta launched its AI support assistant in March, it was presented as a way to help users reset passwords, regain account access, and report problematic content. These are not trivial tasks. Account recovery involves identity, security, and trust—the kind of territory that, not long ago, companies deliberately kept behind a human review process because the consequences of getting it wrong were considered too serious to automate away. Meta has also reassigned thousands of employees into AI-related roles and used AI to automate risk assessments of updates, safety features, and content moderation—all of which are tasks that previously required human review. Therefore, the support bot exploit arrived in an organization that had been replacing human judgment with automated processes across multiple functions simultaneously. Authority without judgement The exploit worked not because the AI behaved unpredictably, but because the AI behaved more or less as it was designed. A user asked it to update a contact email, and it updated the contact email. The system executed a process it was authorized to execute, but in doing so, handed a hacker the keys. This distinction matters enormously for anyone responsible for deploying AI systems. Authority can be automated, but judgment is a different problem entirely. A junior employee in a similar situation might also make a bad call. Humans fail too, but organizations know how to supervise, train, audit, and escalate human decisions. Real people typically offer something an AI system can’t: the ability to notice that something feels off. In this case, hesitation at an unusual request, the instinct to double-check account history, and the recognition that a request arriving via VPN from an unfamiliar location (asking to change contact details on a high-profile account), warrants a pause that could have prevented disaster. Gartner forecasted that 20 to 30 percent of businesses will replace human customer service agents with AI by 2026. The efficiency case for doing so is well established, but what’s less well examined is the category of decisions that look like processes but aren’t. Account recovery looks like a process, but embedded within that process is a judgment call: Is this request legitimate? Humans often make that call poorly too. The difference is that when a human makes it poorly, the failure is legible. There’s accountability. There’s someone to fire. Victims of the Instagram account takeovers told 404 Media there was no way to escalate their problem and speak to an actual human. Historically, high-stakes support requests — account recovery, fraud disputes, identity verification — were escalated to human agents because the potential for harm was too high to leave it to a scripted response. The escalation path was a precaution for the edge cases that automated systems weren’t designed to handle. Research published in the Journal of Consumer Research found that customers evaluate service provided by bots less favorably than identical service provided by humans, partly because they perceive automation as the firm cutting costs at the customer’s expense. That perception hardens quickly when customers discover there’s no human available at all. The harder problem The lesson here isn’t that companies shouldn’t deploy AI in customer service, but the quality of the decisions being made around where AI gets authority and what happens when it exercises that authority incorrectly. The questions that matter aren’t technical; they’re structural. What decisions can AI make autonomously, and which ones require a human to sign off? What triggers an escalation, and who owns it? When the AI gets something wrong — not if, but when — can a customer reach someone who can fix it? Gartner concluded in mid-2025 that half of the organizations expecting to significantly reduce customer service headcount because of AI would abandon those plans by 2027, and that 95 percent of service leaders planned to retain human agents in a digital-first but not digital-only model. The companies that arrived at that conclusion before Meta’s incident are in a better position than the ones who’ll arrive at it after. The Instagram story will probably be remembered as an AI security vulnerability, and will likely get fixed as one. As of June 3, it reportedly still hadn’t been patched. As companies race to automate customer interactions in pursuit of lower costs and faster resolution times, they’re discovering that it’s far easier to reduce the cost of executing a decision than it is to replace the judgment behind it. With better training, retrieval-augmented verification, anomaly detection for suspicious requests, and well-designed human-in-the-loop escalation, AI systems may be able to take on increasingly sophisticated judgment-like tasks over time. But deciding which requests deserve more scrutiny than the process allows remains, stubbornly, a human problem for now. AI can execute authority at scale, but the question that the Instagram incident forces into the open is a simpler one: When it gets things wrong, who’s responsible? Right now, the answer at too many companies seems to be nobody. BY CONNOR JEWISS

How to AI-proof your job

Meta. Nike. Intuit. UPS. It seems every day a new company announces layoffs, often citing AI as the cause. AI is already reducing US monthly payroll growth by roughly 16,000 jobs in the past year, according to a recent Goldman Sachs report. Knowledge workers face the sharpest exposure, as their output is exactly what AI replicates best, at superhuman speed, around the clock. “The most valuable jobs, the ones that we tell people to go to school for – software engineer, finance professional, accountant, lawyer – a lot of these cognitive professions, those are the ones that are the most vulnerable… to AI automation,” David Shrier, professor of AI & Innovation at Imperial College London, told CNN. But humans will always be needed in some form – and there are steps you can take to increase the chances you’ll protect your own job. Audit your role Before you can future-proof your career, you need a clear-eyed view of what you do in your job. Think of jobs as a “collection of tasks we switch between, often many times a day,” Oded Nov, a professor of technology management at New York University, told CNN. Consider which of those tasks are the most repeatable, rule-based computer tasks, like processing expense reports, which takes raw data and converts it into a different form. The more predictable a function, the more vulnerable it is to automation. The CEO of Cloudflare wrote in an opinion piece for the Wall Street Journal that he recently laid off 20% of his work force, focusing on “measurers”: middle management and those who work on audits, operations, compliance, etc. “AI isn’t coming for builders or sellers, but it is coming for measurers,” he wrote. “Tireless, independent, efficient and available, AI systems can now measure an organization with a level of objective detail and precision that was previously impossible even for the best employees.” Some jobs, such as those in hospitality, healthcare and skilled trades, still need someone physically present to do much of the work. Robotics is at least a decade away from replacing those roles. Invest in skills that are structurally hard to automate After your self-audit, focus on skills that are not repeatable, predictable and rule based. In addition to physical duties, AI is not yet as good at handling tasks that require emotional and social awareness, such as “understanding organizational culture or group dynamics,” Nov said. AI tends to be recursive, rather than inventive or creative. “AI is bad at creativity, but it’s surprisingly good at elaborating on creative prompts,” Shrier said. “But you still need the human to come up with the idea and guide the AI to do something interesting.” Invest in those skills. If part of your job involves sales and convincing people to sign a contract, focus on the interpersonal skills that help you build trust with clients. Customers might go to AI to research, but they usually still want to deal with a human being when making big purchases. Embrace AI and learn how to make it work for you AI will soon become a pervasive part of our lives, just like the internet. Get familiar with the major AI systems – ask various AI chatbots about your job and how they can help make your work more efficient, then give their suggestions a try. Play around with new coding tools that help you create your own app and website without needing to write your own code. But chatbots aren’t what makes AI so useful in the workplace, it’s AI agents, or programs that run automatically, make decisions and take actions autonomously. You can learn how to make your own, and a chatbot can help. Try the prompt: “I want to learn how to make an AI agent. Walk me through the steps to create an agent that can do [specific task]”. “In some ways it’s never been a better time to be an entrepreneur, because if you can think of it, you can make it,” Shrier said. “There are people making robust enterprise-grade software that is built off of a prompt in plain English.” Humans can’t be entirely replaced Even roles that include a lot of AI-achievable tasks will still need humans in some way. AI has particularly affected the coding industry. But Anthropic employees still edit and review code even if they’re not writing it, CEO Dario Amodei said at the World Economic Forum in January. “In best-case scenarios, the more mundane tasks that are part of people’s jobs today will be handed over to AI, while the more interesting and rewarding tasks will be done by humans, probably with some support of AI,” Nov said. “It’s very likely, if history is a guide, that new jobs, consisting of new collections of tasks, will be created.” By Hadas Gold

AI ‘voice cloning’ scams are on the rise. Here’s how to protect yourself

A California mom says she was scammed out of thousands of dollars this month after receiving a call that sounded like her daughter in distress. She now suspects it was an artificial intelligence-generated hoax. She’s one of many who have been targeted by so-called “voice cloning” scams as AI tools allow anyone to create a convincing replica of someone’s voice with only a few seconds of real audio. Americans lost more than $893 million to AI-related scams last year, including voice cloning attacks along with AI-generated phishing emails, romance scams and other hoaxes, according to the Federal Bureau of Investigation. Scammers can mimic anyone from family members and friends to coworkers or professional services workers. Banks including the United Kingdom’s Starling and the Commonwealth Bank of Australia have warned customers to watch out for voice cloning scams. Experts say AI voice replicas have gotten so realistic that most people can no longer reliably distinguish them from real human voices. “For the everyday person, it is just not fair to expect them to be able to spot this stuff,” said Henry Ajder, an expert on AI-generated media who consults for governments and companies. “I struggle with it. Most people do.” How do AI voice scams work? Scammers can create an AI replica of someone’s voice using a short recording of their speech — often pulled from social media or an earlier scam call that was surreptitiously recorded. Social media can also provide a trove of information about family members and close friends who could be targeted. Fraudsters will typically make it sound like the loved one they’re mimicking is in distress, purportedly having been kidnapped or in jail. Then they’ll urgently demand money in exchange for their loved one’s release. “There was no time to think,” Gary Schildhorn, a Philadelphia attorney who was targeted by an AI voice scam mimicking his son, told CNN last year. “It was all, ‘I have to react to help my son. He’s in trouble.’” In some cases, the AI voice may be more than just a single recording. Sophisticated attackers could use text-to-speech tools or “voice skinning,” which manipulate a scammer’s voice so they sound like the person they’re imitating in real time. Those techniques facilitate back-and-forth conversations between the target and the AI clone voice, potentially making the scam more convincing, Ajder said. Hackers can also make it appear as if a call is coming from a known number through a tactic known as caller ID spoofing — so you can’t necessarily trust that a call that appears to be coming from your mom is indeed her. How to avoid falling victim to AI voice scams Strange pauses or vocal fluctuations were previously considered red flags that a caller’s voice might be AI-generated. But those signals may no longer be present now that AI has advanced. Instead of trying to determine whether a voice is authentic, look for other general scam warning signs, Hany Farid, a professor at UC Berkeley and chief science officer at GetReal Security, told CNN last year. Is the person on the other end giving a deadline or introducing a sense of urgency? Are they encouraging you not to tell anyone else what’s happening? Are they asking you to move large sums of money in unusual ways? Those are the types of questions experts say to keep in mind. Targets who receive these types of calls should try contacting their loved one through other means, such as via a text message, calling them on another person’s phone or reaching out to someone who would know where they are. Deborah Del Mastro, the California mom recently targeted, told ABC7 News that she called her daughter only after sending money to the scammers. Her daughter answered right away and was at work. Families or coworkers can also establish a precautionary “code word” that can be used to verify each other’s identity. It should be a word or phrase that only a small group of people know and isn’t discoverable online. “Ultimately, if you suspect that something might not be right, it is much better to have your mum or your brother or your friend laugh at you for thinking that they’re a robot,” Ajder said, “than it is to potentially be running to an ATM.” By Clare Duffy

Nobel Prize-Winner Demis Hassabis Says AI Job Cuts Are Dumb. Research Agrees

Across America, graduating college students are booing commencement speakers who mention AI, and communities are battling data centers. A backlash against AI is clearly underway. It’s not hard to see why. With their talk of a jobs apocalypse and general creepiness, plenty of AI CEOs these days don’t seem to realize they often sound like Hollywood supervillains. But there is one notable exception — Demis Hassabis. If you’re not already familiar with him, Hassabis is the head of Google-owned DeepMind. That seems like a position with supervillain potential. But he also won a Nobel Prize in chemistry for his work using AI to solve one of biology’s most vexing problems — predicting the shapes of proteins — and then releasing the solution to the world’s scientists for free. Rather than using AI to replace workers, cheat on everything, or flood the world with slop, Hassabis enthuses about its potential to help cure disease and engineer energy abundance. He’s soft-spoken and bespectacled rather than square-jawed and cape bedecked, and he positions himself as AI’s science-minded good guy. His latest public stance is likely to add to this reputation. As Wired recently reported, he called the recent mania for AI-fueled job cuts basically just dumb. Why AI-driven layoffs are dumb Speaking at Google’s I/O event, Hassabis explained that DeepMind doesn’t look at AI as a way to eliminate jobs and cut costs. Instead, he sees the technology as a way to help businesses dream bigger and do more. “Perhaps there is an ulterior motive for putting those messages out; raising money or whatever,” Hassabis said of other firms slashing headcount supposedly because of AI. “From my point of view, from DeepMind and Google’s point of view, if engineers are becoming three or four times more productive, then we just [want to] do three or four times more stuff.” Sure, companies can use potential productivity gains of AI to trim headcount and pad profits for the benefit of investors. But there’s no shortage of other ways they could deploy the resources freed up by handing many tasks over to the bots. “I have a million ideas, from lab drug discovery to game design,” Hassabis continued. “I’d love to have some free engineers to go and do those kinds of things.” Companies trimming tech talent may be suffering from “a lack of imagination—and a lack of understanding of what’s really going to happen,” he believes. Research shows that layoffs usually backfire Is Hassabis right that companies chasing a short-term boost through AI job cuts may come to regret the decision? We’ll all have to wait for a definitive answer to that question. But there are good reasons to think he might have a point. First, research shows that layoffs in general are often a stupid idea. Of course, they save money. Sometimes they’re genuinely unavoidable (though that mostly doesn’t apply to the mostly super profitable firms doing them now). But multiple studies show that a short-term bump in performance is usually followed by a longer-term decline. Stanford’s Jeffrey Pfeffer summed up the research this way: “Layoffs often do not cut costs. … Layoffs often do not increase stock prices, in part because layoffs can signal that a company is having difficulty. Layoffs do not increase productivity. Layoffs do not solve what is often the underlying problem, which is often an ineffective strategy, a loss of market share, or too little revenue. Layoffs are basically a bad decision.” That’s not even taking into account the human cost to workers. Other research has found that being laid off increases a person’s risk of suicide by two and a half times and increases mortality by 15 to 20 percent over the following 20 years. So why do layoffs? As Hassabis suggests, some firms may be peakcocking for Wall Street. Others may genuinely need to free up funds for a strategic shift. But mostly, Pfeffer insists, “layoffs are the result of imitative behavior. These companies … are doing it because other companies are doing it.” An AI-specific reason layoffs are dumb That’s the general case against layoffs, and it’s pretty strong. But there is another argument against AI-driven layoffs specifically. On the MIT Sloan Management Review website, consultant and author Andrew Winston points out that when firms use AI as an excuse to trim headcount, they also diminish their pipeline of talent. They may save a few dollars. They may also find themselves short of skills and talent when they need them in the future. You can ignore this problem, but it might come back to bite you. As the case of climate change proves, asking CEOs to forego a quick performance bump for longevity and resilience later may be borderline hopeless. Winston continues to point out this inconvenient truth about AI anyway. “I’m asking companies to accept potential (short-term) competitive disadvantages on the basis of uncertain future benefits and collective responsibility. That’s a hard sell,” he acknowledges. “But I have a sneaking suspicion that we will look back at early 2026 and kind of wish we had just stopped.” Don’t get caught up in AI layoff FOMO All of which is a warning to other bosses looking on at tech company layoffs with FOMO. When some of the biggest names in business are all slashing jobs, you might feel like you’re missing a trick if you don’t follow suit. But both Nobel-winner Demis Hassabis and a whole bunch of research suggest leaders should take a breath and really consider whether they want to get swept up in AI job-cut mania. Cutting costs has obvious benefits for your bottom line. But what profit-making possibilities will you not pursue because of those job losses? How will the departure of their colleagues affect the state of mind and productivity of those left behind? Will you likely regret the move later when you find your team is burned out and your bench of experienced talent is incredibly thin? Before you cut jobs, take a minute and consider whether Hassabis is right. Doing more things because of AI could be a smarter choice than employing fewer people. EXPERT OPINION BY JESSICA STILLMAN, CONTRIBUTOR, @ENTRYLEVELREBEL