∞
@warner Phew, thanks, you had me worried there for a bit. 😅 Even if there are efforts from companies to enable some kind of control over how their bots crawl our blogs and websites, I think it's a good to always assume traditional crawlers and AI models gobble up all the data you make publicly available on the open web.
The only way to be reasonable sure they won't see your stuff is to keep it in a more private area of the web, behind a login or paywall, for example. And even then, there's always the risk that some of your readers share your writing with the AI tools they use anyway.