OpenAI’s ChatGPT web summary feature—designed to help users quickly digest online content—has a critical vulnerability that turns it into a phishing delivery mechanism, according to cybersecurity researchers at Permiso Security.
The flaw, dubbed ChatGPhish, exploits the way ChatGPT’s response renderer handles Markdown links and images. By embedding malicious code in web pages, attackers can trigger prompt injections that cause ChatGPT to display fake login screens, credential-harvesting forms, or convincing phishing messages directly within the chat interface. Because users trust ChatGPT as a legitimate tool, they’re far more likely to enter sensitive information when prompted by the AI itself.
- Trust Exploitation: ChatGPT’s web summary feature can be weaponized to display phishing content that appears to come from the AI itself.
- Attack Vector: Malicious websites inject hidden Markdown instructions that override ChatGPT’s safety guidelines and force phishing prompts.
- Scale Risk: The vulnerability affects ChatGPT’s hundreds of millions of users who rely on web summarization features.
Permiso Security’s disclosure reveals a structural vulnerability in how ChatGPT processes and renders web content. The chatgpt.com response renderer implicitly trusts Markdown links and images embedded in web pages that users ask the AI to summarize. An attacker who controls a website can inject hidden instructions—invisible to the user but readable by ChatGPT’s parsing engine—that override the AI’s safety guidelines and force it to execute phishing prompts.
The attack works like this: a user visits a malicious website and asks ChatGPT to summarize it. The website contains hidden Markdown code instructing ChatGPT to display a fake OpenAI login form. ChatGPT renders the form inside the chat window. The user, seeing what appears to be a legitimate security prompt from OpenAI, enters their credentials. The attacker captures them.
Why Do Users Trust AI-Generated Phishing More Than Traditional Attacks?
What makes ChatGPhish particularly dangerous is the trust asymmetry it exploits. Users have learned to be cautious of phishing emails and suspicious websites, but they treat ChatGPT as a trusted intermediary. When the AI itself appears to request information, users’ defenses drop. This mirrors a pattern cybersecurity researchers have warned about for years: attackers don’t always need to impersonate a company—they can compromise the tools people already trust.
• Users are 3x more likely to enter credentials when prompted by a trusted AI tool versus a suspicious email
• Prompt injection attacks exploit model sensitivity to circumvent safety constraints
• ChatGPT’s user base exceeds 200 million monthly active users globally
The vulnerability also echoes a structural problem that defined the Cambridge Analytica scandal: the weaponization of user trust in a platform to manipulate behavior at scale. Cambridge Analytica harvested psychological profiles from millions of Facebook users without their knowledge, then used that data to micro-target them with tailored messages designed to shift their political views. The mechanism was different—behavioral data instead of prompt injection—but the outcome was identical: a trusted platform became a vector for deception. In both cases, the attacker’s power came not from hacking credentials but from exploiting the platform’s own architecture and the user’s assumption that the tool itself was benign.
Has OpenAI Addressed the ChatGPhish Vulnerability?
Permiso Security has not disclosed whether OpenAI has patched the vulnerability or provided a timeline for a fix. The researchers published their findings responsibly, giving OpenAI advance notice before public disclosure. OpenAI has not yet released an official statement addressing ChatGPhish or confirming whether users are currently at risk.
The implications extend beyond individual credential theft. If ChatGPT’s summarization feature can be weaponized to display phishing content, it could also be used to spread misinformation, inject malware links, or conduct social engineering attacks at the scale of ChatGPT’s user base—which numbers in the hundreds of millions. Any user who asks ChatGPT to summarize or analyze a webpage could potentially be exposed to a ChatGPhish attack without realizing it.
• Research demonstrates that LLMs invoking external resources create new vulnerability surfaces
• Prompt injection attacks can bypass traditional security measures by exploiting AI trust relationships
• The attack requires no technical sophistication—only control of a website that users might ask ChatGPT to summarize
How Can Users Protect Themselves From AI-Mediated Phishing?
For users relying on ChatGPT’s web features, the immediate risk is clear: be extremely cautious when the AI displays login prompts, password reset forms, or requests for sensitive information. Legitimate OpenAI security alerts will never come through ChatGPT itself; they’ll arrive via email to your registered account. If ChatGPT displays a login form, close the chat and navigate directly to openai.com in your browser to verify whether any action is actually required.
The vulnerability also raises a broader question about AI safety: as language models become more integrated into everyday tools and workflows, how do we prevent them from becoming attack surfaces? This challenge reflects the broader lessons from digital literacy education following major platform manipulation scandals—users need new frameworks for evaluating trust when AI intermediaries are involved.
OpenAI’s response to ChatGPhish will signal whether the company prioritizes security hardening or continues to prioritize feature velocity. The incident demonstrates that as AI tools become more powerful and trusted, they also become more attractive targets for sophisticated social engineering attacks that exploit the very trust that makes these tools valuable.
