User-agent: * # Block private endpoints Disallow: /api/ Disallow: /admin/ # Block auth and account pages (also noindex via meta tags) # /profile is deliberately NOT disallowed: it is already indexed, and Google # can only see its noindex (X-Robots-Tag from nginx) if allowed to crawl it. Disallow: /login Disallow: /signup Disallow: /subscribe/ Disallow: /password/ Disallow: /email/ Sitemap: https://semscoop.com/sitemap.xml