Ads In Ai: When Does Relevance Become Manipulation?

There were three stories at the intersection of technology, media and marketing last week that, on the surface at least, looked unrelated.

• OpenAI expanded its ChatGPT advertising pilot into more markets.
• Australia’s argument about camera-enabled smart “pervert” glasses intensified.
• And independent MP Allegra Spender continued her push for an ombudsman to support small local businesses whose livelihoods depend on the whims of global digital platforms.

In reality, these are all edges of a bigger story about power. More specifically, a story about what happens when useful technology becomes indispensable, indispensability creates dependency, and that dependency gives one side of a relationship extraordinary power over the other. This is a story emerging in real time, and it’s one those of us in marketing should be paying particular attention to because we are standing unusually close to the machinery while it’s busy changing the world.

Marketers (and pornographers) are usually among the first people invited to turn a new technological capability into commercial advantage. And right now that machinery is learning something advertising has wanted to know for a century: not simply what people want, but why they want it and what that means for what they will soon want (even if they don’t yet know it).

The arrival of paid advertising in ChatGPT will understandably be treated by our industry as another media opportunity. We’ve spent the past year learning the emerging disciplines of organic AI visibility, albeit we still can’t decide whether to call it GEO, LEO or AEO. Now comes paid promoted AI visibility.

So yes, here we go again. Another dashboard, another budget line, another specialist agency arriving with a proprietary framework and a “just comment INORGANIC below to download our white paper”. But the interesting part isn’t the new media inventory on offer. It’s the new information surrounding the moment of persuasion and the safeguards on how that information will be used.

OpenAI says lots of reassuring things. Advertising remains separate from ChatGPT’s answers, is clearly labelled and won’t influence answers. Advertisers won’t have access to users’ conversations or memories. And ads around sensitive conversations will be restricted. All good. But look beyond those safeguards to the adjacent risks.

OpenAI also says the context of a current conversation can be used to determine which ad is relevant and, where users have enabled personalised advertising, past chats and memory can contribute too. That represents a profound shift. Google learned from what we were looking for. Meta learned from what held our attention. Conversational AI can increasingly learn from the whys and worries behind our decisions.

Three thought experiments

Imagine typing “best family SUV” into Google and scanning the first page of results and reviews. Imagine clicking on one Meta ad for a Subaru and now being followed around online for weeks by Subaru and other SUV content. And now imagine spending 20 minutes talking to an AI about the fact that you’re having another child, how your existing car suddenly feels too old and too small, how your partner is planning to take at least a year off work, how money has been tight even before the new baby and how anxious you’re feeling about whether you can afford something safer.

All three user journeys might finish with an advertisement for an SUV and a helpful finance package, but they are not remotely the same advertising opportunity. One reveals intent, another reveals behaviour, while the third reveals context, motivation, trade-offs, anxiety and vulnerability.

Advertising has always tried to understand what makes people persuadable. We invented research, segmentation, targeting, behavioural data and personalisation to get progressively closer to the right person, with the right message, at the right moment. Conversational AI potentially adds something new: the state of mind surrounding the moment. And that creates an uncomfortable question for our industry. At what point does astonishing relevance become an astonishing capacity for informational power and manipulation?

Smart glasses sharpen the point

The same distinction matters in the debate around smart glasses. Much of the current Australian concern understandably centres on recording and consent: can somebody film you without your knowledge, where should camera-enabled glasses be permitted, and what protections should exist? Those questions matter, but AI makes the next question potentially more consequential. Not simply what did the device record, but what did the system infer and recommend someone do as a result?

Wearable technology is moving computing into our field of vision at the same time as multimodal AI is getting better at interpreting images, voices, surroundings and behaviour. The old advertising ambition was the right person, right message, right time. The next frontier could add something else: an increasingly sophisticated understanding of the context surrounding that person and their behaviour in the moment. That is not an argument against the technology. It is an argument for taking the power created by the technology seriously.

Which brings me to Allegra Spender. I recently spoke with Allegra on my new podcast, StoryWork with Matt Jones. Allegra might just be the busiest person in Canberra, and in between talking about tax reform and climate action she’s also campaigning for stronger rules around smart glasses and for better recourse for Australian small businesses when they collide with the digital giants their businesses and livelihoods depend on. That last issue sounds much less futuristic, but I think it’s all part of the same conversation.

An argument for recourse

A small business can spend years building an audience, generating demand and acquiring customers through Meta or Google. These platforms can make the entrepreneur feel like an empowered David taking on big-business Goliaths. Then an automated system makes a decision, an account gets suspended and suddenly the true nature of the relationship becomes apparent. For the platform, you are one account among millions. For the business owner, it may be the customer pipeline, the storefront and the livelihood.

Allegra’s argument for an ombudsman is fundamentally an argument for recourse. If these platforms have become something approaching critical infrastructure for small businesses, what happens when the relationship breaks down and there is nobody meaningful to call? All three stories point towards the same uncomfortable trajectory: useful technology becomes indispensable, indispensability creates dependency, dependency creates power and, at some point, power demands accountability. I think we have reached that point.

For 20 years, digital platforms have made our lives and businesses demonstrably better. The internet was supposed to disintermediate power. Anyone could publish, build an audience, start a business and find customers. And, to an extraordinary degree, it has delivered on that promise. Now AI means that anyone can ask anything, generate anything and level the playing field with anyone. All potentially transformational things. Where there has been a cost, whether in terms of our privacy, our attention and, now and again, our democracy, we have mostly accepted that devil’s bargain.

But along the way we have also built a new generation of intermediaries with enormous influence over whether businesses are discovered, whether content travels, whether customers arrive and, increasingly, what information people encounter when making decisions. AI potentially takes that one stage further. The platform no longer merely mediates connections; increasingly, it mediates decisions.

Not just about labelling

That’s why I’m less interested in whether the sponsored result underneath my ChatGPT answer is clearly labelled than I am in the rules governing everything that surrounds it. What can the system know about us, what can it infer, and what is it allowed to do with that knowledge?

The lesson from the social media era isn’t that technology companies are uniquely malevolent. That’s too simplistic. It’s that commercial incentives have consequences. More relevance creates demand for better targeting, better targeting creates demand for greater understanding and greater understanding creates more opportunities for optimisation.

Marketers will understandably find much of this irresistible. We’ll label it personalisation, relevance and customer experience optimisation. We’ll put a funnel around it and celebrate the conversion uplift. After all, we’ve spent decades asking technology companies to help us understand consumers better.

But that also puts our industry in an unusual position. Marketers are among the first people invited to play with new technology because we’re among the first prepared to pay to turn its capabilities into commercial advantage. We get an unusually early glimpse of where that technology might lead, and I’m beginning to wonder whether AI fundamentally changes the mathematics of the bargain we’ve made with technology. Because these systems won’t simply know more about us. They will increasingly interpret the world for us, influence the decisions we make and, through AI agents, act on our behalf.

We’ve spent most of the past 20 years asking what technology can help us do. Those of us standing closest to the machinery may now have a privileged view of a different question: how much power are we comfortable allowing it to accumulate?

Because the biggest question raised by advertising coming to ChatGPT isn’t whether we’ll be able to tell the paid result from the organic one. It’s whether, as these platforms become better at understanding what makes us persuadable, we’ll retain enough agency to decide what they’re allowed to do with all that knowledge.

Matt Jones is a brand strategist, author, keynote speaker, and entrepreneur. He has worked in British military intelligence, political strategy and speechwriting, and international brand strategy. He returned to Australia to co-found Four Pillars Gin.

Jones’ StoryWork podcast is available on all podcast platforms and his substack. See Spotify, Apple, and Youtube to subscribe.

The post Ads in AI: When does relevance become manipulation? appeared first on Mumbrella.