Learn how to use robots.txt to allow all web crawlers access to your site with an example.
Learn how to block unwanted bots from your website with Robots.txt Disallow All. Example: Disallow: /wp-admin/*