The unprotected mind – The next battle isn’t for content — it’s for how you think
We’ve spent two decades learning to guard our data — our messages, our photos, our passwords. But while we were watching the content, something more revealing slipped past: the behavioural trail underneath it, the pattern of how we think, decide, hesitate and return. The industry has a name for it — profiling — and it’s bought, sold, and, as a 2025 court ruling over ChatGPT logs quietly showed, now reachable by legal process. This is the story of how that trail became the most valuable and least protected thing about us — and why the fix may have to start not with the law, but with us.
The protection paradox
how I was saying it — the patterns, the behaviour, the profile that builds up underneath the words.
Every conversation you have with an AI—every vulnerable moment, every question you ask, every problem you’re trying to solve—leaves a trace. Not of what you said, but of how you reason. Your cognitive patterns. Your decision-making style. Your vulnerabilities and triggers. Your personality structure. These patterns are being systematically collected, analysed, and stored by a handful of companies operating under minimal legal oversight.
And here’s the crucial part: there is no legal framework protecting this data.
Content privacy exists. Cognitive profiling doesn’t. That gap isn’t accidental. It’s where the real power lies, and it’s where we’re most exposed.
The eighties: when behaviour became data
To understand the current crisis, we need to rewind to the moment behaviour first transformed from something personal into something measurable, tradable, and ownable by others.
The 1980s marked the beginning of the end of anonymity in everyday commerce. Credit cards, which had existed since the 1950s, underwent a fundamental transformation. What had been a simple payment instrument became something else entirely: a surveillance device.
As payment cards moved from paper records to fully electronic systems, banks and card processors began collecting transactional data at unprecedented scale. Every purchase told a story. Where you shopped. What you bought. When you bought it. How often. The patterns that emerged—not from any single transaction, but from the aggregate—formed a behavioural map of who you were, what you valued, and what you were vulnerable to.
But the profiling wasn’t limited to banks. Insurance companies recognised that behavioural data could predict risk with striking accuracy. Your purchasing patterns revealed health vulnerabilities. Your location data indicated lifestyle risks. Your spending cycles suggested financial stability or fragility. They could price you differently based on inferred behavioural profiles—higher premiums for those whose behaviour suggested higher risk—without ever asking you directly.
By the late 1980s, companies like Fair Isaac Corporation had built automated credit scoring models based on behavioural patterns extracted from transaction data. These weren’t just assessing your ability to repay debt—they were building predictive models of your future behaviour. Models that determined whether banks would lend you money, at what interest rate, and on what terms.
Retailers, marketing firms, and data brokers entered the picture. Consumer behaviour analytics became a discipline. Behavioural profiles became tradable assets. Companies began buying and selling data about people’s habits, preferences, and vulnerabilities.
Crucially, nobody asked permission. The companies collecting this data owned it. You had chosen to use their services, so they could use the resulting data however they wanted.
This was the template. This was the playbook. Human behaviour had been quantified, standardized, and made commercially tradable for the first time at scale. And it would be refined, scaled, and weaponised over the next four decades.
The social media era: population-level profiling
everything: how long you looked at posts, which ones made you linger, which ones you scrolled past, who you interacted with, when you were most active, what made you angry, what made you laugh.
The behavioural data wasn’t incidental anymore. It was the product.
Streaming and recommendation platforms refined this further. Netflix learned what kept you watching past midnight. Spotify learned what moods triggered which listening habits. YouTube learned which content pulled you deeper into consumption. These seemed like small, convenient features—a personalised recommendation, a curated feed, a suggested next episode. But each one was a piece of a much larger behavioural extraction operation.
Each small data point seemed harmless on its own. Your preference for thrillers over comedies. Your tendency to listen to melancholic music on Sunday evenings. Your habit of watching political content after news cycles. Alone, these pieces appear innocuous. Aggregated across millions of interactions, cross-referenced with behavioural patterns from other platforms and data brokers, they form comprehensive cognitive maps of entire populations.
The platforms didn’t frame this as surveillance. They called it personalisation. It made the experience better for users. In some ways it did. But the trade-off was invisible: your preferences, your patterns, your vulnerabilities became their property.
In 2016, this infrastructure met its moment. Cambridge Analytica emerged as proof-of-concept for what behavioural profiling could actually do at population scale.
Using Facebook data harvested from approximately 87 million users without their knowledge, the company built psychographic profiles based on the Big Five personality model: openness, conscientiousness, extraversion, agreeableness, and neuroticism. Not demographic categories—actual personality maps. Cognitive fingerprints.
Then they weaponised these profiles. Different messages for different personality types. Voters high in neuroticism received fear-based messaging. Voters high in openness received complexity and nuance. Voters low in openness received simplicity and tradition. Each person saw a different political reality, algorithmically tailored to their specific psychological vulnerabilities.
Cambridge Analytica’s executives claimed the company could achieve measurable behaviour change through these micro-targeted campaigns. The company worked across multiple elections and referendums: Brexit in 2016, the Trump campaign in 2016, and dozens of political operations globally.
The legal system had no framework to prevent it. Content regulations didn’t touch it. Privacy laws didn’t cover it. Because what was being extracted wasn’t “personal information” in the traditional legal sense—it was inferred patterns, psychological models, predictions about who you were and how you’d behave. In other words, cognitive property that legally belonged to nobody—which meant it belonged to whoever extracted it first.
A year later, in 2017, Equifax confirmed the scale of the problem. One of the world’s largest credit reporting agencies suffered a breach exposing behavioural and financial profiles of 147 million people. The company had been building these profiles for decades without explicit user consent. Most of those 147 million people had never consciously chosen to do business with Equifax. They couldn’t opt out. They couldn’t access their own profiles. Yet Equifax held detailed behavioural records on nearly half the American population.
The settlement was $575 million. The damage was permanent. And critically—Equifax was prosecuted for negligence in protecting
The AI inflection point: when the legal lock quietly loosened
But here’s where the legal picture shifts in a way that few people are paying attention to.
In 2025, the copyright lawsuit between The New York Times and OpenAI turned, almost as a side matter, into something far more consequential for the rest of us. As part of discovery, a federal magistrate judge — Ona T. Wang — ordered OpenAI to preserve all ChatGPT logs, overriding the company’s own deletion policies for more than 400 million users. Later that year the court went further, ordering OpenAI to hand over twenty million user conversations, anonymised, to the plaintiffs. In early 2026, District Judge Sidney Stein affirmed the order. His reasoning is the part worth sitting with: because users had voluntarily handed their conversations to OpenAI, they couldn’t claim the same protection a person has against, say, a government wiretap.
Read that again. The thing protecting your chats wasn’t a law. It was a promise — a privacy policy. And a privacy policy bends to a court order.
For years, companies reassured users: “We have privacy policies. Your data is protected.” And legally, in some narrow sense, it was — from corporate misuse, from careless breaches, from public exposure. But it was never built to withstand legal compulsion. Let’s be precise about what actually happened here: this was a civil copyright dispute, not an intelligence operation, and the logs were anonymised before they changed hands. No one’s behavioural profile was handed to a spy agency.
And yet — precision is exactly what makes it unsettling. If a copyright case can pry open the conversational records of 400 million people, the principle is now established: this data is reachable through legal process. And a principle, once established, rarely stays in its original lane. What begins as civil discovery has a way of becoming a template — the next argument citing this one, the next demand reaching a little further. The question is no longer whether the cognitive infrastructure of entire populations can be reached through legal mechanism. A court has shown that it can. The question is who reaches for it next, and on what grounds.
What makes this moment different from the Equifax breach or Cambridge Analytica is the depth of the data now at stake.
Payment cards revealed financial behaviour. Search engines revealed informational behaviour—what you were curious about, what you were worried about. Social media revealed social and political behaviour.
But conversational AI reveals something more fundamental: it reveals thinking itself.
When you have an extended conversation with an LLM, you’re not just exchanging information. You’re revealing your reasoning process in real time. How you frame problems. What assumptions you make. Where your knowledge has gaps. How you handle uncertainty. What you’re anxious about. How you construct narratives about yourself and the world.
Every therapy-like conversation with an AI maps your psychological vulnerabilities. Every professional problem you work through with an AI reveals your decision-making style. Every creative project you develop with an AI exposes your cognitive architecture. Every question you iterate on—refining, clarifying, exploring—shows the living structure of your cognition.
The three-layer threat model
Layer 1: Individual vulnerability
At the individual level, every person using an LLM is being profiled in ways they don’t understand and can’t control. You may think you’re having a private conversation with a tool. You’re not. You’re providing raw material for a behavioural profile being constructed in real time.
When you ask an LLM for mental health advice, you reveal psychological vulnerabilities. When you ask for help with a work problem, you reveal professional insecurities and reasoning patterns. When you ask about your relationships, you map your emotional architecture. When you ask how to do anything—you reveal your knowledge gaps, your learning style, your cognitive approach.
The individual thinks: “I have nothing to hide.” But that’s the wrong question. The question is: do you want your thinking patterns, your vulnerabilities, your reasoning style to be known, stored, and analysed by entities you can’t control, optimising for purposes that may not align with your interests?
The asymmetry is profound. The entity holding your cognitive profile understands you better than you understand yourself. It knows your triggers. Your fears. Your decision-making patterns. It can predict how you’ll respond to information, to persuasion, to emotional appeals—before you’re even aware you’re being targeted.
Layer 2: National economic power
At the national level, behavioural profiling creates asymmetric economic advantage. The country whose companies control the largest LLM infrastructure controls cognitive data on a scale unprecedented in human history.
This data enables economic prediction at scale. It enables targeting at scale. It enables influence at scale. A company with access to behavioural profiles of populations across Europe, Asia, Africa, and Latin America understands those markets better than those markets understand themselves. It can predict behavioural responses to pricing, to messaging, to product positioning. It can identify economic vulnerabilities before they become visible.
This is not theoretical competitive advantage. This is structural economic dominance through cognitive intelligence.
Layer 3: Geopolitical control
At the civilizational level, behavioural profiling becomes a tool of statecraft at a scale intelligence agencies could only dream of fifty years ago.
During the Cold War, governments spent billions attempting to understand foreign populations well enough to predict and influence their behaviour. Psychological profiling, cultural intelligence, behavioural analysis—all expensive, slow, and inevitably incomplete.
Now, a handful of companies automatically collect behavioural profiles on hundreds of millions of people continuously, with a depth and precision that dwarfs anything previously possible.
If a nation-state gains access to this data—through legal compulsion, corporate partnerships, espionage, or acquisition—it gains the ability to map the cognitive landscape of entire populations. To identify the most psychologically persuadable segments. To craft messaging calibrated to exploit specific vulnerabilities. To predict which narratives will spread and how to amplify those that serve strategic interests.
This isn’t influence. This is cognitive dominance. And the infrastructure for it already exists.
The historical pattern: a playbook refined over a century
Edward Bernays, nephew of Sigmund Freud and father of modern public relations, understood in the 1920s that if you could identify and trigger unconscious desires, you could change behaviour without people realizing they were being influenced. He applied Freudian theory to mass persuasion—using psychological insights to sell cigarettes to women by reframing smoking as liberation, to shape public opinion during wartime, to orchestrate influence campaigns on behalf of corporations and governments.
Bernays wrote openly: “The conscious and intelligent manipulation of the organised habits and opinions of the masses is an important element in democratic society.”
His work was studied and adopted by those who understood its power. Joseph Goebbels, Nazi Germany’s propaganda minister, was an avid reader of Bernays. He modelled his psychological manipulation campaigns explicitly on Bernays’ techniques, scaling them through the machinery of a totalitarian state.
The Cold War formalized this as doctrine. Psychological Operations—Psy-Ops—became official military and intelligence practice. The goal: understand a target population deeply enough to predict and influence their behaviour without their awareness.
By the 1980s, behavioural science had been commercialised. Marketing companies, data brokers, and financial institutions systematised it for profit. By the 2010s, social media platforms had industrialised it. By 2016, Cambridge Analytica had carried these methods into electoral politics — or claimed to, in operations whose real-world effectiveness remains sharply contested to this day.
Each iteration made the previous one look primitive. And each time, the legal system lagged. The playbook kept working because the frameworks to protect against it didn’t exist yet.
Three scenarios: What comes next
Scenario 1: Uncontrolled escalation
If current trends continue without intervention, behavioural profiling becomes more precise, more comprehensive, and more weaponised. LLMs accumulate billions of hours of cognitive interaction data. Governments establish reliable legal pathways to access it. Companies refine their capacity to predict and influence behaviour based on psychological profiles of unprecedented granularity.
The consequences cascade. Political campaigns achieve perfect psychological targeting. Marketing becomes frictionlessly effective. Dissent becomes predictable and suppressible before it emerges. Information environments fragment completely—different people experiencing different realities algorithmically tailored to their cognitive profiles.
Cognitive autonomy becomes a luxury available only to those with the resources and knowledge to protect it. Everyone else operates in a reality shaped by entities optimising for engagement, profit, or political control. Democracy persists as theater—the appearance of choice without the underlying conditions that make choice meaningful.
Scenario 2: Reactive regulation
In this trajectory, a sufficiently visible scandal forces public and political attention. A government demonstrably uses behavioural profiles to suppress dissent. A foreign power demonstrably uses them to swing an election. A breach exposes the sophistication of what’s been happening invisibly.
Outrage drives legislation. Governments implement frameworks protecting behavioural data similar to existing content privacy regulations. Companies must disclose what they’re inferring about users. Users gain rights to access, correct, and delete behavioural profiles.
This reduces the worst visible harms. But it doesn’t resolve the fundamental problem. Behavioural profiling remains valuable and powerful for those who control it. Compliance costs entrench the largest players while smaller competitors fall away. The underlying infrastructure remains—better regulated, but structurally unchanged.
Scenario 3: Distributed alternatives
In this trajectory, growing awareness of behavioural profiling risks drives genuine market and cultural change. Local and open-source LLMs become viable alternatives. Privacy-preserving architectures make comprehensive profiling technically difficult. Users develop meaningful literacy about what’s being collected and demand real transparency and control.
This doesn’t eliminate profiling—that’s neither realistic nor necessarily desirable. But it redistributes power. Instead of a handful of entities controlling cognitive profiles of billions, multiple providers emerge with multiple approaches and multiple safeguards. No single actor holds comprehensive cognitive maps of entire populations.
Changing the board, not the pieces
This is the part that asks us to change the board, not just move the pieces around on it.
Every time a technology has created a new kind of value, the old categories failed to cover it, and societies eventually built new ones. The printing press gave us copyright. Industrial brands gave us trademark. New inventions gave us patent law. Each arrived because something valuable had appeared that the existing law couldn’t see. We’re at that point again — except this time the valuable thing isn’t a book or a brand. It’s the pattern of how you think.
Privacy frameworks ask: “How do we protect data from being seen?” Ownership frameworks ask something else entirely: “Who has the right to this value, and what can be done with it?”
I know what that distinction looks like in practice, because I used to live by it. For six years I ran a small illustration business, and on the back of every invoice were the licensing terms — what we call leveransvillkor in Swedish. They spelled out, in plain language, what the client was actually buying: not the drawing itself, but a licence to use it in a specific way — this many prints, this context, this purpose. The picture stayed mine. If they wanted to use it beyond what we’d agreed, that wasn’t theft and it wasn’t free; it was simply a new agreement, and a new fee. Ownership and use were two different things, and everyone understood the difference.
That is the model we’re missing for behavioural data. Right now there are no such terms for how you think. The moment you use the service, the profile that forms is treated as theirs — to keep, and to use however they like, with no licence, no bounded purpose, no renegotiation when the use quietly widens. Cognitive ownership is, at bottom, just the radical idea that the same courtesy a freelance illustrator once extended to a client might one day be extended to you.
That shift matters. Your behavioural profile — how you reason, what you fear, how you decide, what you’re vulnerable to — has real economic and political value. It’s extracted from you continuously, used to build products, target advertising, predict behaviour, shape elections. And you receive nothing for it. Most of the time, you don’t even know it’s happening.
So the ownership lens lets us ask better questions. Not “are we protected?” but “who owns this?” Not “is the data secure?” but “do we have the right to extract it in the first place?” Not “what rules should govern its use?” but “should consent and compensation come before any of it is extracted at all?”
None of this is solved. It raises hard complications of its own. What does it even mean to own a cognitive pattern? How would you establish provenance, or compensate millions of people for what they fed into a model? I don’t have those answers.
But the questions themselves are generative. They open territory that privacy frameworks close off, and point toward solutions that regulation alone can’t reach. We don’t yet know what cognitive-ownership law would look like — and that’s exactly the point. New maps are needed. Trying to fit cognitive profiling into privacy law is like regulating digital copying with rules written for physical property: the categories don’t line up, and the solutions don’t fit.
The absent conversation
Who owns your thinking patterns? What can legally be inferred about you from your interactions without your consent? What protections should exist against using cognitive profiles to manipulate your decisions? Who gets access to behavioural profiles, and under what conditions? What recourse do you have if a profile built about you is wrong or used against you? How would you even know?
The legal system has no answers because these questions aren’t yet in the framework. Privacy law protects content. It doesn’t protect cognition.
What needs to happen
So the first move isn’t a policy. It’s seeing the thing plainly: not your words, but the profile of how you think, assembled quietly across thousands of sessions, and valuable precisely because it’s yours.
Then there’s the harder part. Privacy, as we usually mean it, gets defended quickly — because someone powerful has a stake in defending it. Companies protect their data, their assets, their property, because their revenue and their survival depend on it. That pressure is why regulation tends to arrive fast in those areas. Behavioural profiling has no such champion. The people best placed to protect it are the ones profiting from it. There is no company whose survival depends on protecting how you think.
Which leaves an awkward question: who drives this? Who becomes the advocate for keeping your metadata from being turned into a map of your mind? I don’t think it arrives from above, at least not first. I think it has to begin with us — the users, the people whose thinking is quietly being charted. Not because regulation won’t eventually follow, but because this time there’s no one else with a reason to start.
This isn’t a call to arms. It’s just where the reasoning lands. If we want any say in a future being built out of how we think — if we’d rather our own minds weren’t quietly turned into someone else’s asset — then the protecting has to start somewhere. And for once, it starts with us.
And there’s a discipline in the seeing itself. The easy move, when something powerful and new arrives, is to point at the thing and call it the threat. We’re already doing it with AI — it takes our jobs, it manipulates us, it invents our biases. But a tool doesn’t invent a bias; it inherits ours. The skew was in the data, in the documents, in us, long before a model ever read it back — amplified, yes, but never authored. The same holds here. The danger was never the chat box or the helpful assistant. It’s the danger it has always been: someone, somewhere outside the tool, finding new value in us and reaching for it before anyone thought to ask who it belonged to. That pattern is a century old. Only the surface changes.
So awareness cuts two ways. It means seeing the invisible thing — the profile being drawn while you talk. But it also means refusing the lazy verdict. If we want to talk honestly about what’s dangerous in AI, we have to name the threat precisely, not flinch at the tool. Every real advantage arrives with its shadow — and the work is to meet the new clearly enough to tell one from the other.
The real exposure was never the content — the text, the messages, the things we actually write. It’s quieter than that, and easy to miss: the metadata we leave behind about our behaviour: when we show up, what we keep returning to, how we think and how we act. From that, a profile is built — a profile of you.
That is the realisation I keep sitting with. Not what I say, but how I say it. Not the content, but the behaviour underneath it.
Disclaimer
This piece was written in dialogue with AI. The first draft and early research grew out of a conversation with Claude, then it was refined, fact-checked and edited in the Stimulus workflow — including a structured integrity pass using the STIMULUS SENTINEL method. The reasoning and conclusions are my own.
Opinion. Stimulus.se is a personal exploration, not a publication of record. The views here are mine, offered as thinking-in-progress.
Sources. Key references: the OpenAI / New York Times litigation and the 2025–26 ChatGPT-logs discovery rulings (SDNY); reporting on Cambridge Analytica and the Big Five personality model; the 2017 Equifax breach; and Edward Bernays’ Propaganda
0 Comments