Every major AI company wants two things at once. They want to read your website. And they don't want you reading theirs.
That contradiction is shaping the agent era more than any model release or protocol announcement. In my last post I argued that the Personal Agent Protocol is cute but will fail. This is the other half of the story: why something like it was always going to show up, and why it's so hard to get right.
King of the hill, powered by hypocrisy
Start with the crawlers.
Every AI provider is racing to pull the freshest data off the internet to train and ground its models. The internet, reasonably, is fighting back. Publishers block crawlers. Sites hide content behind logins, paywalls, rate limits, and CAPTCHAs. Robots.txt files grow longer every month.
So the AI providers get creative. Residential proxies. Rotating IPs. Headless browsers dressed up to look like a person on a laptop in Ohio. Whatever it takes to get the data out.
Now look at those same companies' own products. Try scraping a chatbot's web app. Try automating an AI provider's dashboard. You'll hit bot detection, fingerprinting, rate limiting, and terms of service that forbid exactly the kind of automated access they rely on everywhere else.
The rule, in practice, is simple: my bots get through your walls, and your bots stay out of mine.
That's not a technical stance. It's king of the hill. Whoever is on top kicks down the ladder they just climbed. And a huge share of what's driving this paradigm shift is that game, played with a straight face, by everyone at once.
The baby and the bathwater
Here's the collateral damage.
When everyone is crawling aggressively and everyone is defending aggressively, websites stop asking whether a bot is good or bad. They just ask whether it's a bot.
And from the outside, good and bad bots look nearly identical. The scraper vacuuming up your content and the personal agent trying to book my dentist appointment both show up as automated traffic from an unfamiliar browser. One is stealing. The other is doing a chore for a paying customer. The site can't easily tell the difference, so it blocks both.
The good bot gets thrown out with the bathwater because the bad bots made the bathwater filthy.
That is the real tension behind things like the Personal Agent Protocol. Businesses want a way to say yes to the agents acting on behalf of real people, without saying yes to every scraper on earth. It's a fair problem. It just isn't an easy one.
We have seen this movie before
Remember the early days of Windows.
You could download an .exe from anywhere, double-click it, and install it. That openness was magic. Anyone could ship software to anyone. No gatekeeper, no review, no permission.
It was also how you got a virus.
The same openness that let a kid in a dorm ship the next great utility let someone else ship malware that wiped your hard drive. Open by default meant dangerous by default.
Agents are in the same spot. The whole promise is frictionless: tell your assistant what you want, and it goes and does it across any site, any service, any account. But frictionless for good actors means frictionless for bad ones too. Every open standard eventually meets the people who abuse it.
Do we need an app store for bots?
The PC world's answer was the app store. Curated, signed, reviewed software, largely to keep viruses and malware off people's machines. It worked. It also handed enormous power to whoever ran the store.
So here's the question for the agent era: do we need an app store for bots?
A registry of approved agents. Signed identities. Verified intent. A trusted lane for good bots and a locked door for everyone else.
That might be exactly what the Personal Agent Protocol is reaching for, whether or not anyone says it out loud. And if so, it comes with the same trade-off app stores did: safety, in exchange for a gatekeeper.
It shouldn't have to be this way
Here's where I land.
If we actually believe AI is the magical thing we keep saying it is, a smart agent should be able to prove it's acting for a real person, behave like a good guest, and get on with the job, without a consortium of trillion-dollar companies deciding who's allowed through the door.
The app store fixed viruses by building a wall. I'd like to think AI can do better than another wall.
But as long as the biggest players keep crawling everyone else's hill while fortifying their own, good bots are going to keep paying for the sins of bad ones. And protocols like PAP will keep looking less like an open door and more like a velvet rope.
No comments yet