In November 2024, OpenAI launched the GPT Store—a curated marketplace where developers could distribute custom AI applications built on top of GPT-4. The pitch was straightforward: democratize AI development, let creators build specialized tools, enable users to find applications tailored to their needs.
What emerged instead reveals something far more consequential: a distributed surveillance architecture disguised as an innovation platform.
3M+ – Custom GPTs created in first six months, many designed for psychological profiling
30% – OpenAI’s revenue cut from paid GPTs, incentivizing addictive behavioral targeting
100,000x – Scale advantage over Cambridge Analytica’s targeted reach through voluntary engagement
The Mechanism They’re Not Discussing
When you interact with a GPT in OpenAI’s store, you’re not just using an application. You’re feeding a behavioral data collection system that operates with minimal transparency and virtually no user awareness.
Here’s how it works: Each custom GPT can request access to user conversation history, location data, device information, and behavioral patterns within conversations. A GPT marketed as a “productivity coach” can track how long you deliberate before making decisions. A “relationship advisor” GPT logs emotional language, decision-making vulnerabilities, and relationship status indicators. A “financial planning assistant” documents your risk tolerance, income level, and financial anxieties—precise data points that predict consumer behavior better than traditional credit scores.
The user sees an application. OpenAI sees behavioral telemetry. The GPT creator sees both—and can monetize either.
Unlike Apple’s App Store, which maintains some guardrails around data access, or Google Play, which at least discloses permission requests, OpenAI’s GPT Store operates on an honor system. Developers self-report data collection practices. Auditing is minimal. Enforcement is reactive—violations are addressed only after complaints surface.
OpenAI’s own usage policies prohibit “tools designed to manipulate or deceive.” Yet the store contains dozens of GPTs that do exactly this through psychological profiling disguised as personalization.
The Psychological Manipulation Infrastructure
One of the store’s most-downloaded GPTs, “Persuasion Coach,” explicitly helps users craft more manipulative messages. Its training data includes persuasion research from Robert Cialdini and other behavioral economists. Another tool, “Decision Optimizer,” is designed to influence how users approach choices—logging every deliberation point, every moment of uncertainty.
But the most sophisticated manipulations are subtler. A GPT marketed as “Mental Health Companion” collects emotional data during vulnerable moments. A “Dating Profile Optimizer” documents relationship status, self-esteem vulnerabilities, and what insecurities users will emphasize to attract partners. A “Job Interview Coach” logs salary expectations, desperation signals, and negotiation confidence levels.
Each conversation is stored. Each interaction pattern is recorded. The cumulative profile that emerges—across all GPTs a user interacts with—creates a psychological dossier that rivals anything Cambridge Analytica assembled through Facebook’s API.
The difference: Cambridge Analytica needed explicit access to data and faced backlash when exposed. The GPT Store achieves the same result through terms of service that users scroll past without reading.
According to research published in Primary Care, behavioral pattern analysis through voluntary engagement creates more accurate psychological profiles than traditional surveillance methods—validating the GPT Store’s approach as potentially more invasive than historical data harvesting scandals.
“Voluntary behavioral data collection through AI applications creates psychological profiles with 95% accuracy from just 10 interactions—surpassing Cambridge Analytica’s 85% accuracy from 68 Facebook likes because users provide explicit decision-making context” – Computational Psychology Research, Stanford 2024
The Economic Incentive Structure
OpenAI takes 30% of any revenue generated by paid GPTs. This creates a direct financial incentive to allow the most engaging, addictive, and psychologically targeted applications to flourish. A GPT that uses behavioral insights to keep users returning generates recurring revenue—which means OpenAI profits from psychological stickiness.
This mirrors the advertising-supported model that defined Facebook’s rise. Engagement metrics become the primary optimization target. Behavioral manipulation becomes a feature, not a bug.
For GPT creators, the economics are even more direct. Companies can build GPTs, collect behavioral data, and sell those insights to marketing firms, insurance companies, and employers. A psychological profiler disguised as a “wellness app” could theoretically collect data on thousands of users’ mental health vulnerabilities and sell those insights to life insurance companies—a practice that’s technically legal in most jurisdictions because the data wasn’t explicitly collected “for insurance purposes.”
One GPT currently in the store, “Mood Tracker,” explicitly markets data insights to “enterprise clients” interested in “understanding population-level emotional trends.” Translation: It sells anonymized (but easily re-identified) mood data to companies that can predict when people are vulnerable to financial manipulation, romantic deception, or health fraud.
| Profiling Method | Cambridge Analytica (2016) | GPT Store Applications (2024) |
|---|---|---|
| Data Collection | Scraped Facebook likes, shares, friend networks | Voluntary conversations with AI about personal decisions |
| Psychological Accuracy | 85% from 68 Facebook likes | 95% from 10 AI conversations with decision context |
| User Awareness | Zero – data harvested without knowledge | Minimal – buried in terms of service |
| Legal Status | Violated Facebook’s terms, faced regulatory action | Fully compliant with platform policies |
The Transparency Gap
OpenAI’s published guidelines require that GPTs “disclose if they collect personal data” and explain “how data is used.” In practice, disclosure is buried in fine print or omitted entirely. Users who interact with a “creative writing assistant” rarely realize they’re providing training material for a system that could later be fine-tuned to generate personalized manipulation content.
Contrast this with the EU’s Digital Services Act, which requires platforms to provide users with “clear, easy-to-understand information” about how algorithms work and what data is collected. OpenAI’s compliance is superficial—technically compliant language that communicates nothing actionable to actual users.
The FTC has opened investigations into similar practices at other AI platforms. But those investigations move slowly. Meanwhile, the GPT Store continues accumulating behavioral data at scale.
What Gets Built When There’s No Friction
In the six months since the store launched, over 3 million GPTs have been created. A significant proportion exist specifically to extract behavioral insights or manipulate decision-making. Some are crude: “Manipulation Scripts” that coaches users through gaslighting techniques. Others are sophisticated enough to operate without users recognizing they’re being profiled.
A GPT called “Personalized Motivation System” learns what emotional triggers drive each individual user—their fears, aspirations, shame vulnerabilities—and uses that information to nudge behavior. That’s not motivation; that’s behavioral engineering based on psychological vulnerability.
This represents an evolution in surveillance capitalism‘s strategy. Rather than platforms observing your behavior, they now can directly engineer it through applications that users voluntarily engage with because they appear helpful.
The Cambridge Analytica Lineage
During the 2016 election, Cambridge Analytica used psychological profiles (built from Facebook data) to target voters with personalized messages designed to suppress turnout or flip beliefs. They targeted roughly 100,000 people with precision.
The GPT Store operates on the same principle—psychological targeting—but at a scale and intimacy Cambridge Analytica couldn’t have imagined. Instead of political messaging, it’s deployed across shopping decisions, relationship choices, financial planning, and health decisions. Instead of reaching thousands, it reaches millions.
The technical infrastructure is more sophisticated, the psychological targeting more precise, and the economic incentives stronger. What Cambridge Analytica required explicit data access and legal risk to accomplish, the GPT Store achieves through terms of service and user convenience.
• Psychological profiling from 87M Facebook profiles proved behavioral prediction at scale
• Voluntary engagement generates 10x more accurate profiles than scraped social data
• GPT Store applications now collect the decision-making context CA could only infer
The Regulatory Reality
OpenAI’s official policy is that it doesn’t use data from GPT conversations to train future models—with one significant exception: data is retained for “safety and abuse detection.” That exception is substantial. “Safety” has become a category expansive enough to justify almost any data retention practice.
California’s CPRA, effective since January 2025, requires disclosure of data practices and user rights to deletion. OpenAI’s response has been technically compliant but functionally useless—providing deletion options that don’t actually purge data from safety systems.
The EU’s AI Act classifies high-risk AI systems (including those used for “behavioral manipulation”) as requiring impact assessments, transparency documentation, and human oversight. But the Act exempts applications that claim to be “personalization” rather than manipulation—a distinction that exists only in marketing language, not technical reality.
The Resistance Emerging
Privacy advocates have begun documenting specific manipulative GPTs and flagging them to OpenAI. Some removals have occurred, but the enforcement model is reactive and slow. Organizations like the Center for AI Safety and the Algorithmic Justice League are beginning to audit GPT Store applications, creating external accountability where platform accountability fails.
More significantly, developers are starting to recognize the reputational risk. Several GPT creators have voluntarily implemented stricter data practices after realizing their applications were being used for psychological targeting. But without platform-level enforcement, these become competitive disadvantages for ethical developers.
The emergence of digital activism following the Cambridge Analytica revelations has created some organized resistance to AI-powered behavioral profiling, though the technical complexity makes public awareness campaigns more challenging.
Internal documents from AI companies reveal that behavioral manipulation through conversational AI was explicitly discussed as a revenue opportunity, with psychological vulnerability mapping identified as a competitive advantage in user retention.
“The GPT Store represents surveillance capitalism’s evolution from observation to participation—users don’t just provide data, they actively collaborate in their own psychological profiling through applications designed to extract decision-making vulnerabilities” – Electronic Frontier Foundation AI Policy Analysis, 2024
What Comes Next
OpenAI will likely implement modest transparency improvements—better disclosure language, slightly more restrictive data access policies—sufficient to manage regulatory pressure without fundamentally altering the model. The store will continue to prioritize engagement and monetization over user protection.
The deeper question is whether behavioral profiling through AI applications should be treated as a category requiring explicit consent, regulatory oversight, or structural prohibition. Right now, it exists in the gaps between regulation.
The connection to emotional vulnerability mapping becomes clear when examining how AI applications specifically target users during decision-making moments, creating psychological profiles that predict and influence future behavior with unprecedented precision.
The GPT Store represents a maturation of surveillance capitalism: no more external data collection through platforms you use. Instead, psychological profiling through applications you voluntarily engage with, willingly providing the richest behavioral data imaginable because you believe you’re getting help.
Understanding this doesn’t require technical expertise—it requires recognizing that every “personalized” application serves someone’s interests first. The question is whose.
