Forum being down/AI crawlers.

[Steve Goodey]

Found this very interesting.

This is the bit that really annoys me.

In the face of bots riding roughshod over polite opt-outs like robots.txt directives,

How can any reputable company do this? Creating work for website admins and taking sites down? It's terrible.

[white_haired_uncle]

Found this very interesting.

This is the bit that really annoys me.

In the face of bots riding roughshod over polite opt-outs like robots.txt directives,

How can any reputable company do this? Creating work for website admins and taking sites down? It's terrible.

Reputable?

Meta accounted for more than half of all AI crawler traffic on its own, at 52 percent, followed by Google and OpenAI at 23 percent and 20 percent respectively.

The recent changes to mythtv.org have made a world of difference.

[Steve Goodey]

And thanks to David for that work.