Google's AI push meets the UK's scraping block

Google is accelerating AI-driven advertising while UK regulators now let businesses stop AI from harvesting their online content.

Abstract violet flow-field artwork

Google is accelerating AI-driven advertising while UK regulators now let businesses stop AI from harvesting their online content. The result is a strategic paradox, you can embrace Google's AI ad tools to boost performance, but if you also block the same AI from learning about your brand, you may vanish from AI-generated search results and limit the tools' effectiveness. The immediate priority is an AI data audit, a clear robots.txt strategy and a structured test of Google's AI Brief before your competitors do.

On a single day this week, Google told advertisers they could finally take manual charge of AI-generated ad creative, while UK regulators told every business it could tell the same AI to keep its hands off their website data. This is not a contradiction to file away. It's a fork in the road that will define which brands appear in AI search, and which disappear, over the next 18 months.

Google's new AI Brief, a steering wheel for ad creative

Think with Google detailed a new tool called AI Brief, designed to give advertisers control over AI-generated images and copy. The pitch is straightforward, brands feed in a brief, product, tone, visual direction, and the AI produces multiple assets, but with guardrails the advertiser sets. It's a nod to the fact that Performance Max and Demand Gen campaigns have often left creative directors feeling like passengers. AI Brief puts copy and image controls in one interface, aiming to cut down the volume of off-brand outputs that frustrate in-house teams.

Separately, Google Business Profile's coverage of "Infinite marketing in the age of agentic AI" paints a larger picture, AI agents that not just generate assets but autonomously plan, buy and optimise campaigns. The language is future-tense but the product roadmap is concrete. For businesses, this signals a coming shift where marketing strategy and AI governance must merge, because the agent will make real-time budget decisions without human sign-off.

The UK's new scraping block, publishers push back

Meanwhile, Real Business reports that the UK's Competition and Markets Authority has handed UK businesses the right to instruct platforms not to scrape their content for AI training. The enforcement framework means Google, among others, must respect opt-out signals embedded in a site's robots.txt or served via specific meta tags. The rule change, part of the CMA's wider investigation into digital advertising, effectively gives media owners, retailers and SaaS companies a legal shield they lacked, and a commercial dilemma.

That dilemma is immediate, if you block AI scraping, you protect proprietary product descriptions, editorial IP and customer data. But you also risk vanishing from AI Overviews and any surface where a large language model synthesises answers from the open web. Google has not clarified whether a blocked page can still appear in a generated answer. Early evidence suggests it cannot.

The paradox, control the inputs, lose the outputs

Here the two stories collide. Google is betting its AI tools will dominate creative production and media buying. AI Brief and agentic marketing both assume the AI has learned enough about a brand's product catalogue, tone and performance history to produce decent work. If a business blocks the very scraping that feeds that learning loop, the tools become blunt instruments. An agent that can't ingest your latest product feed or tone-of-voice page will default to generic copy that undercuts conversion rates.

For search visibility, the trade-off is starker. AI Overviews already drive a measurable share of organic clicks for informational queries. Opting out of AI training may mean your content becomes invisible in that real estate, ceding the spot to a competitor who allowed scraping. Advertisers who rely on AI-generated creative in Performance Max will find their campaigns less competitive if the same AI can't understand their product detail pages.

How strong is your own AI marketing? Take the free two minute scorecard and get an honest read plus the next steps that matter most.

Take the scorecard

The CMA's move also opens a new front in the agency-client conversation. Clients will ask whether their agency's "AI-powered creative" was built on data they didn't consent to share, and whether the agency's own site blocks scraping. Procurement teams will add AI data usage to pitch questionnaires. The transparency burden has shifted from regulator to practitioner.

Action this week, audit, decide, test

Marketing leads should treat this as a prompt to run a rapid AI data audit. First, check your robots.txt and response headers, is your site currently allowing the handful of AI crawler user agents? If you block them, do so deliberately and document the risk. Second, map which pages you can afford to lose from AI Overviews and which are critical for organic discovery. Product pages, FAQs and editorial that rank for high-intent queries should be protected at the crawling level only if you have a strong alternative traffic source.

On the ad side, request access to AI Brief and run a controlled experiment. Use a small budget, a simple product set and strict brand guidelines. Measure not just click-through rate but the ratio of approved to rejected assets. That ratio will tell you whether the tool, in its current form, reduces creative churn or simply adds another layer of review. Finally, brief your legal and compliance teams on the CMA's new opt-out rules. Being proactive now avoids a scramble when a partner or client asks for your AI data policy.

Key takeaways

  • Google's AI Brief gives advertisers manual control over AI-generated ad images and copy, reducing the risk of off-brand outputs.
  • The UK's CMA now requires platforms to honour "AI scraping" opt-out signals, letting businesses block Google's AI from training on their content.
  • Blocking AI scraping can protect intellectual property but may remove you from AI Overviews and limit future AI-driven ad performance.
  • Agentic AI, as Google describes it, will soon create and manage entire campaigns autonomously, a shift that requires new governance frameworks.
  • Every UK business should review its robots.txt and site-level AI directives this month, then test AI Brief in a controlled campaign.

Common questions

What is Google's AI Brief and how does it help advertisers?

AI Brief is a creative control tool that lets advertisers set specific instructions, product details, visual style, tone, for AI-generated ad images and copy. It aims to reduce off-brand outputs and give marketing teams more say over the creative that appears in Performance Max and Demand Gen campaigns.

Can UK businesses legally block Google's AI from scraping their content?

Yes. The Competition and Markets Authority's new rules require major platforms to respect opt-out signals. Businesses can block AI crawlers by updating their robots.txt file or using specific meta tags, and the CMA framework provides a legal basis to enforce this block.

Will blocking AI scraping hurt my visibility in AI-generated search results?

Likely yes. If your content is inaccessible to AI crawlers, it probably will not appear in AI Overviews or similarly synthesised answers. You could lose organic traffic from informational queries, so the decision should be weighed against the risk of IP exposure.

What is agentic AI in marketing and why does it matter?

Agentic AI refers to AI systems that autonomously plan, execute and optimise marketing campaigns with minimal human input. Google's recent publication signals this is moving from concept to product. It matters because it will require new governance, transparency and data-sharing policies inside organisations.

Put this to work on your business

We build exactly this, SEO, AI visibility, PPC, email and automation, as one measured system. Talk it through with a real person, no obligation.

Book a free strategy call or take the two minute scorecard