SEO sitemaps as a first class citizen

[AlemTuzlak]

SEO support

This proposal aims to simplify the SEO story in Remix by making generation of sitemaps easier and more convinient. It is a bit hard at the moment to generate a sitemap without hacking into Remix internals, hence why I'm proposing this.

Benefits

• Better overal SEO performance for websites OOTB
• Better Sitemap generation
• Easier access to server data while generating sitemaps
• Easier robot.txt generation

Background

There are libraries like:

• https://www.npmjs.com/package/@nasa-gcn/remix-seo
• https://github.com/balavishnuvj/remix-seo

That aim to solve this problem already in similar ways by accessing either the EntryContext in the entry.server.tsx or the routes export from @remix-run/dev/server-build which I'm pretty sure isn't a thing in the Vite land but I might be mistaken. These are not sustainable long-term due to the reliance on internals which might go away at some point, some sooner rather than later!

In order to generate sitemaps you need to know which routes the app has, when was the route last modified, what's the priority etc. This can all be done server-side without touching the client. The only thing you need to be vary off is dynamic routes and url segments which would need a way to get this information before returning the full manifest of the available routes.

Proposal

This would depend on how much Remix wants to offer out of the box. The easiest for the user land would be the following:

{
  future: {
    unstable_seo: true;
  }
}

This would setup the sitemap.xml and robots.txt files with sensible defaults and you can create the files yourselves to override these like with entry.server and entry.client files. Another approach would be to create the sitemap and maybe inject it into context, eg:

// sitemap[.]xml
import { generateSitemap } from "@remix-run/seo";

export const loader = ({ context }) => {
  return generateSitemap(context.sitemap);
}

In Vite land this could run when you initially run the project and on sitemap export changes with HMR. Maybe there's even a smarter and easier way to do it that I'm not thinking of, would love to have a discussion on the topic here. These were the things that came to mind as potential solutions.

The general idea would be that each route module has a sitemap export which allows you to define entries for the final sitemap output, and the route module would look like this:

// Export this function if you want to define sitemap entries, otherwise remix would just resolve 
// to the current path as entry with default values
export const sitemap: SitemapFunction = ({ request, data }) => {
   return  [
    { url: "/blog/1", lastModified: Date.now(), priority: 1 },
    ...
  ];
}

If you want to opt-out of generating a sitemap for the children of the current route (sitemap export in a child route would override this):

export const sitemap: SitemapFunction = ({ request, data }) => {
   return false;
}

If the route doesn't export anything and the parent routes haven't disallowed sitemap.xml entries the current route url would be used as a sitemap entry.

Open questions

• How much does Remix (if at all) want to help with sitemap generation?
• What would be the best way to generate this data? Server Context RFC would allow database injection etc into sitemaps, but would it make sense to be called on request?
• Use cases I have missed? Are there any use-cases that people run into that I haven't documented? I would add them here as updates to the proposal.