An XML Sitemap lets you share data about the pages on your website with search engines. By doing so, search engines gain deeper understanding of which pages exist on your website that they could possibly include in the search results. XML sitemaps become increasingly important for larger websites where it is more likely search engines will be unable to crawl to every page on your website without a comprehensive list of all pages—that comprehensive list is the XML sitemap.
There are also specific XML sitemap formats for news, images, and videos. These can be especially helpful if your website relies heavily on that type of content. For example, if you regularly share videos on your website, having a video XML sitemap can make it easier for search engines to index those videos.
When building your XML sitemap, the biggest mistake made is including pages you don’t want Google to find or pages Google simply shouldn’t find. For example, XML sitemaps that include broken URLs, URLs containing low-quality content, duplicate content, or URLs that are blocked (either in the robots.txt file or via a meta tag) will return errors and won’t do much to help your SEO performance.
Creating an XML sitemap is relatively straightforward and the protocol is fairly simple if you need to create an XML sitemap manually. There are also XML generation tools that can automatically create your sitemap. As well, many plugins exist that allow you to create XML sitemaps dynamically, including the Yoast SEO plugin for WordPress.
After creating the XML sitemap, you want to tell Google and Bing about your new sitemap. The easiest way is to submit your XML sitemap directly to Google and Bing via Google Search Console and Bing Webmaster Tools.
In Google Search Console, click on Crawl, then click on Sitemap.
From here, you can click on "Add/Test Sitemap", enter the URL to your XML sitemap, then click "Submit Sitemap." Following this, Google will begin reviewing your XML sitemap.
You can check back to this same page later to see if Google has detected any errors with your XML sitemap. One of the first things to review is the difference between the pages submitted in the XML sitemap and the number of pages indexed. If there are a lot of pages on your XML sitemap not included in Google’s index, this is an indication you might have submitted pages in the XML sitemap you shouldn’t have. Review your XML sitemap for error URLs, low-quality pages, duplicate pages, or similar issues and remove those from the XML sitemap (and potentially, consider removing those pages from your website too).
Google will also tell you if there are issues within your XML sitemap. Clicking on the count of issues (in the example below, clicking on the link that reads “8,606 warnings”) will give you a list of example issues Google encountered within the XML sitemap. Reviewing these issues and correcting them can often help improve the way Google crawls your website and improve your website’s SEO performance.
In Bing Webmaster Tools, navigate to Configure My Site, then click on Sitemaps.
After arriving here, enter in your XML sitemap URL, then click submit. Bing will then crawl through your XML sitemap. You can check back to this same page later to see if Bing has detected any errors with your XML sitemap.
Another way is to list the XML sitemap on your robots.txt file. This is especially helpful for when your XML sitemap is not available at the standard "domain.com/sitemap.xml" URL or when you are unable to submit the sitemap to Google Search Console or Bing Webmaster Tools. For example, here is the XML sitemap directive on our robots.txt file:
Want help improving your website’s technical SEO factors? Contact us today to discuss how we can help review and improve your current technical structure.