This is the bit that really annoys me.
How can any reputable company do this? Creating work for website admins and taking sites down? It's terrible.In the face of bots riding roughshod over polite opt-outs like robots.txt directives,
Forum being down/AI crawlers.
Moderator: Forum Moderators
-
Steve Goodey
- Moderator
- Posts: 247
- Joined: Fri Feb 07, 2014 6:30 pm
- Location: Colchester, England
Forum being down/AI crawlers.
Found this very interesting.
This is the bit that really annoys me.
This is the bit that really annoys me.
Don't forget the Wiki.