In this post, we see a couple of ways to set up Astro Sitemaps. First, we use the Astro sitemap integration. Then we see how you get more fine-grained control by creating your own sitemaps on resource routes. On top, we see adding these and even custom styling are not at all difficult. Before we get into that, though, we take a look at why XML sitemaps are important and also which fields are needed in 2022. If that sounds like what you came here for, then let’s crack on!
XML sitemaps are great for Search Engine Optimization (SEO) as they provide an easy way for search engines to determine what content is on your site and when it was late updated. This last part is essential as it can save the search engine crawling a site which has not been updated since the last crawl.
Crawling is the process by which search engines discover sites and also attempt to ascertain what a particular page is about. When crawling, the search engine bot looks for anchor tags and uses its existing data about the site linked to, as well as the text between the opening and closing anchor tag. These are used to work out what your site and the linked site are all about. Anyway, the crawl is all about finding links and updating the search engine’s index. Sites get allocated a budget which will vary based on a number of factors. This caps the time a search engine will spend indexing your site. You risk the search engine recrawling existing content without discovering and indexing your new pages if you don’t have an up-to-date sitemap.
In the example sitemap content above, we see three entries. Each has
loc tag which contains the URL of a site page as well as
the last modification date for the page. Hence, by scanning this sitemap the search engine will be
able to work out which content you updated since the last index. That saves it re-crawling
unchanged content. On a larger site, bots might discover fresh content quicker.
On older sitemaps, you might see
changefreq tags. Although search engines used these in the past, they no longer matter much to Google . For that reason, we will skip those tags in the rest of this post.
Astro integrations let you quickly add certain features to your site and typically need little or no configuration. Here we see how to set up the Astro sitemap integration. If you already know you need something more sophisticated, skip on to the next section.
Like other integrations, the
astro addcommand helps you get going quickly on the sitemap integration:
When prompted, type
Yto accept installing the integration and also placing necessary config in your
sitefield in your
astro.config.mjsto match your site’s domain:
As a final step, you can update the HTTP headers for the final Astro sitemap (more on this
later). The method will depend on whether you are building your site an SSG
Static Site Generatedsite (Astro default) or SSR Server-Side Rendered.
To see the sitemaps, you need to build your site normally (
pnpm run build), then run the preview server (
pnpm run preview). You can see
the sitemaps at
http://localhost:3001/sitemap-0.xml. The first is an index which will just links to the second. This second one includes dynamic
pages like posts (depending on your site structure). You can also inspect these two files in the
You will notice there are no dates by default. It is pretty easy to add a
lastmod date parameter, though this will be the same date
for all pages . If you want something more sophisticated, you can supply functions in the configuration, which
generate the data you want. My opinion is that for this use case, it makes more sense to add a resource route with your own custom XML sitemap. This should keep the sitemaps more maintainable. We see this in the next section.
We will add three sitemaps, though for your own site you might decide to go for more or even
fewer. All the sitemaps will include
lastmod tags to help search
engines optimize indexing. The first sitemap will be an index with links to
posts sitemaps. The index one is easiest with the least
dynamic data, so let’s start there.
Astro does not just create fast HTML pages; you can also use it to create resource routes. We see
how to serve JSON data and even PDFs in the Astro Resource route post, so take a look there for
further background if you are interested. Let’s start by creating a
src/pages/sitemap_index.xml.js file. This will generate the content served when a search engine visits
This code is based on the Astro Blog Markdown starter, and we will get information on post modification dates from the Markdown front matter for blog posts. The two entries (pages and posts) have the same last modified date because the home page includes a list of recent posts, so we assume the content there gets updated each time a blog post is updated. Because we are adding the logic ourselves, you can modify this better to suite your own use case if this is a poor assumption.
We add our sitemap to a
get function, since this is the method the
search engine will use to access it. We are assuming static generation here. If your site runs in
SSR mode, consider adding an HTTP
content-type header (see Astro
resource routes post for details).
8 above, the
vary depending on whether we are running the site locally in dev mode or in production mode. We
use Astro APIs to get the last date a post was updated, pulling data from post metadata. The full
code is on the Rodney Lab GitHub repo, see link further down the page. Most important, for your
own project, is the code in lines
30 with the XML markup. You can even add
images and videos in here if you want to.
We also included an XSL stylesheet in line
20 just to make the
site look a bit nicer for you, while debugging! Create a
public/sitemap.xsl file with this content:
public/sitemap.xsl — click to expand code.
This is based on code in an Alexander Makarov GitHub repo . Try opening the Sitemap in your browser. It should look something like this:
Next up, here is the page code. The update dates are a bit more manual here. You have to remember to update them manually when you update content. An alternative is using the file modified date, though this can be complicated when using continuous integration to deploy your site. Note that the index sitemap links to this one.
We repeat the logic to get the last post update date. In your own project, you will probably want
to move that code to a utility function if you also need it twice. In lines
22 we have a manually compiled list of
site pages (excluding dynamic post routes, which we add to their own sitemap in the next section).
For each page, we include the path and
lastModified date. Then we
use this array to generate the output XML.
Finally, our posts will have dynamic dates, using logic similar to what we saw earlier, to get
last modified fields from post Markdown front matter. Here is the
That’s it! Check out the new sitemaps in the browser.
Search engines do not need to index the sitemaps, so you can serve a robots
noindex directive from these routes. There are a few ways to set this up. If you are use SSG (Astro
default) and hosting on Cloudflare or Netlify, then to let the host know which headers you want to
include, add a
public/_headers file to the project:
However, if you are running in SSR mode, then you can just include these headers in the
Response object which your
get function returns.
In this post, we saw how to add Astro Sitemaps to your project. In particular, we saw:
- how to use the Astro sitemap integration for hassle-free setup,
- how you gain more control over the sitemap content using Astro Sitemaps XML resource routes,
- serving noindex HTTP headers on sitemap routes.
You can see the full code for the project in the Astro Blog Markdown GitHub repo.
Hope you have found this post useful! I am keen to hear what you are doing with Astro and ideas for future projects. Also, let me know about any possible improvements to the content above.
- Yes, there is an Astro sitemap integration. It makes it quick and easy to add a basic sitemap to your site. To set it up, just run `pnpm astro add sitemap` from the Terminal. The Astro add tool will prompt you on installing the plugin and updating your config. You can answer yes to both. Next, just update the `site` field in your Astro configuration in the `astro.config.mjs` file. Although it is easy to set up a basic sitemap with the integration, if you want more control, for example over page modified dates, you can consider serving your own custom Astro sitemap from a resource route.
- Yes! And although Astro has a sitemap integration, unless you need a fairly basic sitemap, you might find your code becomes more maintainable and intuitive if you create your own custom sitemaps and serve them from resource routes. Doing so, you can add logic to derive the last modified date for each page. As an example, we saw for a Markdown blog, we could pull the last updated date from the post front matter for each post. Doing this, we better reflected the update date, hopefully helping search engines to crawl the site more efficiently.
- Astro makes it easy for you to add resource routes. These let you serve a file or some data instead of the regular HTML, Astro is famous for serving at speed! You could go for an XML sitemap, an RSS feed, or even a PDF of the company newsletter. You would expect this in Server-Side Rendered mode, but note, Astro lets you create resource routes even in the default, Static Site Generator mode. To get going create a source file to match the path of the resource (just add .js or .ts on the end). Then export a `get`, `post` etc function which returns the resource data.
Have you found the post useful? Would you prefer to see posts on another topic instead? Get in touch with ideas for new posts. Also, if you like my writing style, get in touch if I can write some posts for your company site on a consultancy basis. Read on to find ways to get in touch, further below. If you want to support posts similar to this one and can spare a few dollars, euros or pounds, please consider supporting me through Buy me a Coffee.
Just dropped a new, free post on adding a sitemap using the integration and also the pro mode resource route alternative.— Rodney (@askRodney) November 16, 2022
We look at styling, which tags to include in 2022 and also serving noindex directives.
Hope you find it useful!
Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on Twitter, @[email protected] on Mastodon and also the #rodney Element Matrix room. Also, see further ways to get in touch with Rodney Lab. I post regularly on Astro as well as SEO. Also, subscribe to the newsletter to keep up-to-date with our latest projects.