An XML sitemap is one of the oldest tools in SEO and one of the most consistently misconfigured. Most guides make two mistakes: they tell you to fill in tags Google stopped using years ago, and they imply that submitting a sitemap gets your pages indexed. Neither is true, and clearing both up is where real sitemap skill begins.
This is the full deep-dive into Step 1 of the technical SEO strategies guide on GrowWithSakib. If you’re still getting your bearings, what technical SEO actually is on GrowWithSakib sets the scene.
What an XML Sitemap Actually Does (and Doesn’t)
Held correctly, that idea reframes everything. The sitemap’s job is to make discovery efficient and to give you a diagnostic surface in Search Console. It is not a magic “index me” button. Everything below serves those two real purposes.
What to Include – and What to Exclude
The golden rule: a sitemap should contain only the URLs you want indexed, and nothing else. Every junk URL you include dilutes the signal and wastes crawl budget. A good test – if you’d be unhappy seeing a page in Google’s results, it doesn’t belong in your sitemap.
| Include (canonical, indexable) | Exclude (dilutes the signal) |
|---|---|
| Key landing and service pages | Tag pages and thin taxonomy archives |
| Blog posts and articles | Author archive pages (especially single-author blogs) |
| Products and product categories | Internal search results pages |
| Important static pages (About, Contact) | noindex pages of any kind |
| Cornerstone and pillar content | Redirected URLs (list the destination instead) |
| Canonical version of each page | 4xx / 404 and blocked (robots.txt) URLs |
The Technical Limits You Must Respect
| Rule | The Limit | What to Do |
|---|---|---|
| URLs per file | 50,000 maximum | Split into multiple files beyond this |
| File size | 50 MB uncompressed | Split, or compress with gzip |
| When you exceed either | – | Use a sitemap index file pointing to child sitemaps |
| URL format | Absolute URLs only | Full https:// URLs, never relative paths |
| Location | Root is best | yourdomain.com/sitemap.xml, not a subfolder |
What a Sitemap Actually Looks Like
A basic sitemap is simpler than people expect. Each URL needs a <loc>; an accurate <lastmod> is a useful bonus:
| <?xml version=”1.0″ encoding=”UTF-8″?> <urlset xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″> <url> <loc>https://example.com/</loc> <lastmod>2026-07-10</lastmod> </url> <url> <loc>https://example.com/services/</loc> <lastmod>2026-06-22</lastmod> </url> </urlset> |
For a large site, a sitemap index points to multiple child sitemaps, usually split by content type:
| <?xml version=”1.0″ encoding=”UTF-8″?> <sitemapindex xmlns=”http://www.sitemaps.org/schemas/sitemap/0.9″> <sitemap> <loc>https://example.com/sitemap-posts.xml</loc> <lastmod>2026-07-10</lastmod> </sitemap> <sitemap> <loc>https://example.com/sitemap-pages.xml</loc> <lastmod>2026-06-22</lastmod> </sitemap> </sitemapindex> |
How to Create Your Sitemap (WordPress and Beyond)
You almost never build a sitemap by hand. The method depends on your platform:
WordPress: Rank Math or Yoast
Both plugins auto-generate a sitemap index (usually at /sitemap_index.xml) and split it into child sitemaps for posts, pages, categories, tags and custom post types. They update it automatically whenever you publish or edit. The setup that matters is what you turn off:
- Rank Math: Sitemap Settings, then disable the tag, author, and any thin taxonomy sitemaps you don’t want indexed. Toggle post types on or off individually.
- Yoast SEO: under Search Appearance, set author archives, tags, and formats to “noindex” – Yoast then automatically removes them from the sitemap.
- Either way: the key insight is that noindex and sitemap inclusion are linked in these plugins – mark something noindex and it drops out of the sitemap, exactly as it should.
For deeper plugin configuration, see the WordPress SEO setup guidance on GrowWithSakib.
Other Platforms
- Shopify, Wix, Squarespace: generate a sitemap automatically at /sitemap.xml – nothing to install.
- Next.js / static sites: use a build-time generator (like the next-sitemap package) and exclude private paths such as /admin/*.
- Custom sites: generate programmatically from your database, including only canonical 200-status URLs. Validate with a crawler like Screaming Frog before deploying.
How to Submit Your Sitemap to Google
Once your sitemap is clean, submitting it in Google Search Console on GrowWithSakib takes about a minute:
- Find your sitemap URL (on WordPress, usually yourdomain.com/sitemap_index.xml). Open it in a browser to confirm it loads.
- In Google Search Console, open Indexing, then Sitemaps in the left menu.
- Enter your sitemap path in the ‘Add a new sitemap’ field and click Submit.
- Reference it in robots.txt too, so every crawler finds it: add the line Sitemap: https://yourdomain.com/sitemap_index.xml
- Check back in a few days: Search Console will report ‘Success’ and how many URLs it discovered.
The Real Diagnostic: Submitted vs Indexed
This is where the sitemap earns its keep, and where the “discovery, not indexing” idea pays off. In Search Console’s Pages report (and the per-sitemap view), you’ll see two very different numbers: how many URLs you submitted, and how many Google actually indexed. The gap between them is a diagnosis.
| What You See | What It Means | What to Do |
|---|---|---|
| Submitted approx equals indexed | Healthy. Google values your pages | Nothing – this is the goal |
| Many ‘Crawled – not indexed’ | Google saw them but judged them low-value | Improve content depth, quality, uniqueness |
| Many ‘Discovered – not indexed’ | Google is holding off crawling (often crawl budget) | Improve site speed, internal links, reduce bloat |
| ‘Excluded by noindex’ | Correct IF intended; a bug if not | Check the page shouldn’t be indexed |
| Sitemap ‘Couldn’t fetch’ | Google can’t read the file | Check the URL, format, and robots.txt access |
The two most common – Crawled currently not indexed and Discovered currently not indexed – are not sitemap problems. They’re content and site-quality signals, and worth checking as part of a full technical SEO audit on GrowWithSakib. No sitemap edit fixes them; better, more distinct content and stronger internal linking do. That’s the sitemap doing its real job: not indexing your pages, but telling you why they aren’t.
Keeping It Healthy
- Let it update automatically – a static, hand-maintained sitemap is stale within days. Dynamic generation is the default for any site that publishes regularly.
- Keep sitemap and robots.txt in agreement – never list a URL in your sitemap that robots.txt blocks. That contradiction wastes crawl budget and confuses Google.
- Match canonical URLs exactly – every sitemap URL should match that page’s canonical tag, character for character (trailing slashes included).
- Audit quarterly – and always after a migration, plugin change, or URL restructure. That’s when sitemaps quietly break.
Common XML Sitemap Mistakes
| Mistake | Why It Hurts | Do This Instead |
|---|---|---|
| Including tag/author/search pages | Dilutes the signal, wastes crawl budget | Exclude them; list only index-worthy URLs |
| Setting changefreq and priority | Google ignores both entirely | Skip them; use accurate lastmod only |
| Faking or uniform lastmod dates | Google ignores lastmod site-wide | Use the real content-modified date |
| Listing noindex or redirected URLs | Sends Google contradictory signals | List only canonical, 200-status URLs |
| Sitemap and robots.txt disagree | Contradictory crawl signals | Never list a URL robots.txt blocks |
| Expecting a sitemap to force indexing | It only aids discovery | Fix content quality for ‘not indexed’ pages |
| Never checking Search Console | You miss the submitted-vs-indexed gap | Read the Pages report regularly |
| One giant file over 50,000 URLs | Exceeds Google’s hard limit | Use a sitemap index with child files |
Frequently Asked Questions
What is an XML sitemap and do I need one?
An XML sitemap is a file listing the URLs on your site you want search engines to discover and index – a structured table of contents you hand directly to Google. You need one because it helps crawlers find your content efficiently, especially on new sites, large sites, and pages with few internal links pointing to them. Even a small, well-linked site benefits: a sitemap costs nothing, speeds up discovery of new pages, and gives you a valuable diagnostic surface inside Google Search Console.
What should I exclude from my XML sitemap?
Exclude anything you wouldn’t want appearing in Google’s results. The usual culprits on WordPress are tag pages (thin taxonomy archives), author archives (which duplicate your blog feed on single-author sites), and internal search results. Also exclude any noindex pages, redirected URLs (list the destination instead), and anything returning a 404 or blocked by robots.txt. The rule is simple: a sitemap should list only canonical, indexable, 200-status URLs you genuinely want indexed. Everything else dilutes the signal.
How do I submit my sitemap to Google?
Open Google Search Console, go to Indexing then Sitemaps in the left menu, enter your sitemap path (on WordPress usually sitemap_index.xml) in the ‘Add a new sitemap’ field, and click Submit. Then add a line to your robots.txt file – Sitemap: https://yourdomain.com/sitemap_index.xml – so every crawler can discover it automatically. Check back after a few days: Search Console will report success and how many URLs it found. Referencing it in robots.txt also lets Bing and AI crawlers find it without separate submission.
Why are my sitemap pages submitted but not indexed?
Because a sitemap only aids discovery – it doesn’t force indexing, which Google decides independently. If Search Console shows pages as ‘Crawled – currently not indexed’, Google saw them but judged them low-value, usually due to thin or duplicate content; the fix is better, more distinct content, not a sitemap edit. If they’re ‘Discovered – currently not indexed’, Google is delaying crawling, often a crawl-budget issue you address with faster pages, stronger internal links, and less site bloat. The gap between submitted and indexed is a content signal, not a sitemap fault.
Should I use changefreq and priority tags?
No. Google has stated openly that it ignores both the changefreq and priority tags, and has for years – it crawls based on its own signals rather than your self-assigned values. Adding them creates clutter and a false sense of control. The one metadata tag still worth including is lastmod, and only when it reflects the genuine date a page’s content last changed. If you want to signal which pages matter most, do it through internal linking and site structure, which Google actually weighs.
What is a sitemap index file and when do I need one?
A sitemap index is a master file that points to multiple child sitemaps, rather than listing individual URLs itself. You need one when your site exceeds a single file’s limits – 50,000 URLs or 50 MB uncompressed – or simply when you want to organise URLs by type for easier diagnosis. Most large sites split into child sitemaps for posts, pages, products, and categories. You submit just the index file to Search Console, and Google discovers all the child sitemaps from it. WordPress SEO plugins create this structure automatically.
Rank Math or Yoast for WordPress sitemaps?
Both generate a clean sitemap index automatically and update it whenever you publish or edit, so either works well. The practical difference is in configuration. In Rank Math, you control the sitemap directly under Sitemap Settings, toggling post types and disabling tag or author sitemaps. In Yoast, sitemap inclusion is tied to indexing settings: mark author archives, tags, or formats as ‘noindex’ under Search Appearance and Yoast removes them from the sitemap automatically. Choose based on your wider plugin preference; for sitemaps specifically, they’re comparable.
How often should I update my XML sitemap?
Ideally never by hand – it should update automatically. On WordPress, Rank Math and Yoast regenerate the sitemap the moment you publish, edit, or delete a page, so new content appears within minutes. A static, manually-maintained sitemap goes stale within days and defeats the purpose. What you should do periodically is audit it: check quarterly, and always after a migration, plugin change, or URL restructure, since those are exactly the moments a sitemap quietly breaks or starts including URLs it shouldn’t.
Key Takeaways
- An XML sitemap aids discovery, not indexing. It tells Google which URLs exist and are worth crawling – but Google still decides indexing independently.
- Include only canonical, indexable, 200-status URLs. Exclude tag pages, author archives, search results, noindex pages, redirects, and 404s.
- On WordPress, tag and author pages are the biggest culprits – left in by default, they can outnumber your real content and bury it.
- Google ignores changefreq and priority entirely. Don’t set them. Use an accurate lastmod – and only ever the genuine content-modified date.
- Faking or uniformly stamping lastmod makes Google ignore the field across your whole site. Honesty is the whole point of it.
- Keep each file under 50,000 URLs and 50 MB. Beyond that, use a sitemap index pointing to child sitemaps split by content type.
- Submit in Search Console and reference it in robots.txt with one line – that lets Google, Bing, and AI crawlers all discover it.
- The submitted-vs-indexed gap in Search Console is the real payoff: ‘Crawled – not indexed’ is a content-quality signal, not a sitemap fault.
