Back to Discovery
Lesson 1
How Search Engines Find Websites
Why this matters
If search engines can't crawl, index, or understand your pages, your customers never see you. This is the foundation of every visibility decision you'll make.
Crawling
Bots follow links from page to page, discovering URLs. New sites need at least one inbound link OR a submitted sitemap to be discovered.
Indexing
After crawling, the engine decides whether to store the page. Pages with thin content, duplicates, or noindex tags are skipped.
Ranking
Stored pages are scored per query. Relevance, authority, freshness, and user-experience signals decide order.
The four assets that drive discovery
These four files/tags do 80% of the work:
- •Sitemap.xml — tells engines what exists
- •Robots.txt — tells engines what to skip
- •Internal links — distribute authority
- •Titles + meta descriptions — control how you appear in results
Learn More
All lessons
1 of 10