How to Extract All URLs from an XML Sitemap (3 Easy Ways)

Updated July 25, 2026

🛠️ This guide pairs with our free Sitemap URL Extractor — Pull every URL out of an XML sitemap or sitemap index, with lastmod dates and free CSV export.

An XML sitemap is the closest thing to a complete, official list of a website’s pages — the site itself publishes it so search engines know what to crawl. That makes it the fastest starting point for a site audit, a migration checklist, a competitor content inventory, or a crawl list for a scraper. The problem: sitemaps are XML built for machines, often split across dozens of child files and sometimes gzip-compressed. Here are three reliable ways to turn one into a clean URL list.

First, find the sitemap

Two places cover almost every site:

  1. https://example.com/sitemap.xml — the conventional location.
  2. https://example.com/robots.txt — look for a Sitemap: line, which is where sites declare non-standard locations like /sitemap_index.xml or /wp-sitemap.xml (WordPress).

If the file you find contains <sitemapindex> instead of <urlset>, you’ve found a sitemap index — a sitemap of sitemaps. Large sites split their URLs across many child files (the format caps each file at 50,000 URLs), so you’ll need a method that follows those children automatically.

Method 1: browser tool (no code, ~10 seconds)

The quickest route is our Sitemap URL Extractor: paste the sitemap URL, click Extract, and it recursively follows every child sitemap in an index, decompresses .xml.gz files, and gives you the full list with lastmod dates — filterable with plain text or regex, exportable to CSV, TXT or JSON. Parsing runs in your browser, so there’s no server queue, no URL cap and no signup.

This is the right method when you want the list now: auditing your own sitemap for stray URLs, sizing up a competitor’s content, or building a one-off crawl list.

Method 2: Python (repeatable, scriptable)

When you need this weekly or want the URLs flowing straight into a pipeline, a dozen lines of Python does it:

import requests
from xml.etree import ElementTree

NS = {"sm": "http://www.sitemaps.org/schemas/sitemap/0.9"}

def extract(sitemap_url):
    xml = requests.get(sitemap_url, timeout=30).content
    root = ElementTree.fromstring(xml)
    # If it's an index, recurse into each child sitemap
    for child in root.findall("sm:sitemap/sm:loc", NS):
        yield from extract(child.text.strip())
    for url in root.findall("sm:url/sm:loc", NS):
        yield url.text.strip()

urls = list(extract("https://example.com/sitemap.xml"))
print(f"{len(urls)} URLs")

requests transparently handles gzip transfer encoding, but a sitemap stored as a .xml.gz file needs one extra line: xml = gzip.decompress(xml) when the URL ends in .gz.

Method 3: command line (curl + grep)

For a quick-and-dirty list on a machine with standard Unix tools:

curl -s https://example.com/sitemap.xml | grep -oP '(?<=<loc>)[^<]+'

This regex-extracts everything between <loc> tags. It won’t follow sitemap indexes — pipe the first result back through the same command for each child — but for a single flat sitemap it’s unbeatable for speed.

What to do with the URL list

Troubleshooting

Try the free Sitemap URL Extractor →