robots.txt is a text file at the root of a site that tells crawlers which paths they may or may not fetch.
Why it matters
It is a request, not security, and it does not reliably remove pages from search results. Use noindex for that.
Source: RFC 9309