Robots.txt Generator
Choose a preset or build custom rules per crawler, then generate a ready-to-use robots.txt file.
Upload the generated file to your site’s root directory as /robots.txt.
Choose a preset or build custom rules per crawler, then generate a ready-to-use robots.txt file.
Upload the generated file to your site’s root directory as /robots.txt.