Prioritize creating a sitemap if your website is large or has a complex hierarchy, is new and lacks external links, or contains substantial image, video, or news content. A sitemap provides URLs and update information to help search engines discover pages. You can start with XML, list the full URLs and last modification dates of public pages, add multimedia information as needed, and specify the sitemap’s location in robots.txt.
First, consider how easily pages can be found
Search engines usually discover a website’s pages through internal links. A sitemap becomes more useful when a site has many pages or a deep hierarchy, making it difficult to ensure that every page is reachable through links. News and e-commerce sites can also use sitemaps to provide content update information, but website type alone should not determine whether one is needed.
- Large or structurally complex websites: A sitemap helps search engines discover pages that are difficult to find through links.
- New websites lacking external links: A sitemap provides a list of URLs for search engines to discover.
- Sites with substantial multimedia or news content: A sitemap can provide content-related information in addition to URLs.
An introductory iThome article describes sites with “no more than 500 pages, comprehensive internal linking, and no plans to display multimedia or news content on Google” as potentially not needing a sitemap. These are assessment criteria relayed by that article; 500 pages should not be treated as a current hard cutoff for whether a sitemap is required.
Build the URL list first, then add update information
Where feasible, consider XML first: it supports extensions for images, videos, news, and localized versions. If you only need to list indexable URLs, a text format is simpler. For websites using a CMS, first check whether the system already generates an RSS, mRSS, or Atom feed.
- loc: Enter the page’s full URL, such as https://www.example.com/about.
- lastmod: Specify the page’s last update date to help search engines decide whether to crawl it again.
- Image and video information: Add this as needed for the content; video information can include a title, description, duration, and thumbnail URL.
Do not treat priority as a command specifying crawl order; setting priorities does not guarantee that search engines will crawl pages in that order.
Declare the sitemap’s location and plan file splitting around site growth
Once you have an XML sitemap, you can declare its location with a Sitemap directive on a separate line in robots.txt. The Next.js implementation article shows how to create sitemap.js at the root of the app directory to generate XML. The URL list includes only public pages, not administrative pages.
If you expect the number of pages to grow quickly, consider the suggestion in the developer Q&A: have the generator create an appropriate number of files based on the total number of pages being submitted. Large websites can also organize them by month or season. If you will not need multiple files anytime soon, start with a basic list and address splitting as that need approaches. There is no need to design a complex file-splitting system from the outset.
Sitemaps: perspectives from across the web
Can multilingual websites use a sitemap to specify hreflang?
Yes. A sitemap can list a page’s different language or regional versions; another approach is to declare them in each page’s HTML. Include the page’s own version and ensure that corresponding versions reference one another. For non-HTML files such as PDFs, you can also consider declaring them through HTTP headers. This is an alternative, not a required sitemap setting.
How should you set last modification dates in a dynamically generated sitemap?
Use each page’s actual last update time. You can retrieve it from the content data and add it to the corresponding URL entry in the sitemap. The Next.js example uses the update times in user data to generate these entries dynamically. Last update dates can help search engines decide whether to crawl a page again. The example also shows an approach that uses the current time, but you should not simply use the sitemap’s generation time as every page’s actual modification time.
References
- How do you configure robots.txt to allow crawling of the site except for a few directories?webmasters.stackexchange.com — Source used in the article above.
- Hreflang是什麼?如何規劃多語系與多地區網站?www.da-vinci.com.tw · 2023-05-14T21:53:59+08:00 — Discusses: Can multilingual websites use a sitemap to specify hreflang?
- Day7:網站地圖 Sitemapithelp.ithome.com.tw — Discusses: How should you set last modification dates in a dynamically generated sitemap?
- Sitemaps index submissionwebmasters.stackexchange.com — Source used in the article above.
- 營養師不開菜單的第二十七天 - Next.js SEO 極致優化,為網站提升曝光度!ithelp.ithome.com.tw — Discusses: How should you set last modification dates in a dynamically generated sitemap?



