Glossary

Crawling

The process by which search engine bots discover and fetch web pages by following links.

What is Crawling?

Crawling is the process where search engine bots (like Googlebot) discover web pages by following links and fetching content. Crawl budget determines how many pages get crawled.

Key Takeaways

  • 1.Crawling is important for website performance and SEO.
  • 2.You can measure it using tools like BulkAudit and Google Lighthouse.
  • 3.Improving this metric can lead to better user experience and search rankings.

Learn More

Google Web Fundamentals

Related Terms

Related Guides

Browse Full Glossary

Explore all SEO and web performance terms

Test Your Crawling Score

Use BulkAudit to measure crawling and other key metrics for your website.

Start Free Audit