User-agent: * Disallow: /wp-admin/ Disallow: /test/ Allow: /wp-admin/admin-ajax.php # Affiliate click-out interstitials. /slot-sites//play/ is a meta-refresh # page that lands on the operator tracking URL, so every crawler fetch books an # affiliate click that can never convert. The page already carries # noindex,nofollow, but that only stops indexing - it does not stop the fetch. # Checked against all 9,090 URLs in the sitemaps: none contain /play/, so nothing # indexable is affected. # NOTE: /terms/ is deliberately NOT blocked - those pages carry real content. Disallow: /*/play/ # The same problem, two more routes. Both are meta-refresh interstitials that land on # an operator tracking URL, so every crawler fetch books an affiliate click that can # never convert - and unlike /play/ these are linked heavily from inside the site: # SEMrush counted 27,437 internal links to /offer/ alone, across every attribute page # (/software/*, /payment-method/*, /withdrawal-method/*). They already carry # noindex,nofollow, which stops indexing but not the fetch. # /slot-sites//offer// the offer click-out # /terms//1/ the operator T&C click-out # /terms// itself is a 404 and /terms//2/ is a real page, so the rule # is pinned to /1/ rather than the whole /terms/ tree. Disallow: /*/offer/ Disallow: /terms/*/1/ Sitemap: https://www.slotfruit.co.uk/sitemap_index.xml