Google’s John Mueller and Martin Splitt recently discussed how HTML sitemaps may still be useful for SEO even though the XML standard has largely replaced it.
Pre-XML Sitemap Era
Before the introduction of XML sitemaps, SEOs and site owners typically added internal and external links to new web pages to ensure that they got crawled and started ranking.
They also added a link from the footer of the home page (or every page) to an HTML sitemap where a gigantic list of links to inner pages could be crawled and aid in getting those pages indexed. Most SEOs populated the HTML sitemap with every page on their site that was considered useful to rank. Other pages, like the About Us pages, were left off of the HTML sitemap.
It’s possible that Google was already using RSS feeds well before the introduction of XML sitemaps. For example, a 2004 university research paper documented that RSS feeds could be used by web crawlers to identify new and recently modified web pages and reduce crawling bandwidth by up to 40%.
But because RSS feeds weren’t universally implemented, they were likely impractical to use for crawl discovery. That’s probably why GYM (Google, Yahoo, and Microsoft) joined forces to agree on a single XML sitemap standard that is used to this day.
HTML Sitemaps Not A Replacement For XML Sitemaps
During the course of their recent Search Off The Record episode, Martin Splitt asked if SEOs and site owners were locked into using XML sitemaps OR if it was okay to use HTML sitemaps.
Google’s Mueller explained that HTML sitemaps are not a replacement for XML sitemaps because they lack the structural features inherent in XML sitemaps.
“Ah yes, HTML sitemaps. That I think is very confusing to the average person because you probably know what XML is and what HTML is, but a lot of people when they open an XML file, they see those weird brackets and they’re like, oh, this is HTML or very similar, right?
It looks like this weird programming stuff. And they’re called the same, but an HTML sitemap is basically almost like a map of your website for users. It’s not something that replaces an XML sitemap file.
Of course, if someone is crawling your website and they find an HTML sitemap file, they can crawl those links like any other link on a website, so that’s helpful.
But it’s not the case that the HTML sitemap can be submitted as a sitemap file. It can’t be processed one-to-one the same way as an XML sitemap file because it doesn’t have this strict structure to it. It’s basically a collection of links.”
Google Explains How To Use HTML Sitemaps
Somewhat surprisingly, Google’s Mueller explained that HTML sitemaps can still be used. His answer implied deploying them as a part of a navigational system for users. Mueller used the example of an ecommerce site, but his answer can apply to any kind of website.
In his example, an HTML sitemap enables site visitors to easily reach the top-level product (or topic) category pages from which site visitors can drill down to a product (or topic) they are interested in.
His answer was in response to Martin Splitt’s question:
“But that still would help crawling because from this place we can find all the links as well. It’s just not as structured and clear cut as an XML sitemap, I guess.”
“Yes, absolutely. And it’s something I think that a lot of HTML sitemaps, they don’t include everything.
So if you have an e-commerce site, you wouldn’t list all of your products in an HTML sitemap file. You would list maybe the categories so that people can go to the right place, but then pick out the product individually. So it’s similar, but not exactly the same.”
Should You Use HTML Sitemaps For SEO?
For some readers, this is a relatively new idea. For others, the concept of an HTML sitemap might be perceived as old and outdated like directory links or reciprocal link arrangements. And yet, Google’s John Mueller says that they can be useful for users, which means HTML sitemaps could be useful for Google.
If you’re going to do it, do it in a way that’s useful for users, and you may find that site visitors will start using and recommending more pages from the site, and Google will pick up on those signals.
Takeaways
- HTML sitemaps have a long history in SEO. Before XML sitemaps, they helped search engines discover pages and were used to distribute PageRank.
- HTML sitemaps are not replacements for XML sitemaps. Google explains that they serve different purposes, although both can support crawling.
- Google confirms HTML sitemaps are still useful. They can improve website navigation for users while helping search engines discover pages.
- HTML sitemaps don’t need to contain every page. Google suggests organizing them around useful navigation paths rather than creating enormous collections of links.
- The SEO value of HTML sitemaps starts with users. Making websites easier to navigate can also benefit crawling and content discovery.
Watch Google’s Search Off The Record at the 17 minute mark:
Featured Image by /Aleksandra Bataeva
