---
title: Making my blog readable for AI agents
description: I added Markdown versions, an agent index, and build checks to every
  post. Not as a GEO trick, but to make the underlying source less ambiguous.
canonical: https://djangodevreng.nl/en/blog/making-my-blog-readable-for-ai-agents/
pubDate: '2026-08-18T00:00:00Z'
category: build-logs
---


An agent could already read my blog. The content is ordinary HTML, like almost every other blog. The direct Markdown routes and HTML alternates already existed too. But that is different from giving a blog a complete machine-readable entry point. An agent that wants to use a post as a source should not first strip navigation, layout, and scripts, then guess where the main content starts.

In this change, I completed those existing parts as one contract: a central index for every language, consistent sitemap freshness, and checks against the built site. Not as a GEO trick. I want an agent to find the same source as a reader, with a clear canonical URL and a date.

That fits the reason I build this site and [the Arena](/en/blog/why-this-blog-and-arena/): I do not only publish a conclusion, but also the path that led to it. For benchmarks, that means commands, runs, and failures. For a blog, it means the source itself should not be needlessly vague for software that has to retrieve it. The same thinking sits behind the [Local AI decision guide](/en/local-ai/): make the trade-off explicit first, then reach the conclusion.

## HTML works for people, but it is not the shortest path for an agent

A browser needs HTML. A reader sees a header, navigation, images, code blocks, and the rest of a page as intended. An agent usually needs less: the text, title, date, URL it can cite, and preferably a way to discover which posts exist before fetching individual pages.

[Vercel's guide to documentation for AI agents](https://vercel.com/kb/guide/make-your-documentation-readable-by-ai-agents) separates that into discovery, retrieval, and tool access. For this blog, retrieval was the first practical win: an agent should be able to fetch Markdown without extracting the HTML page first.

This does not make an agent smarter or guarantee correct answers. (Note: The benefit is smaller and concrete: the same content, an explicit format, a canonical URL, and freshness metadata.)

I do not use content negotiation or user-agent detection here. The site is static, and the explicit route is simple: the HTML page lives at `/blog/slug/`, and the machine-readable variant at `/blog/slug.md`. (Note: That keeps caching and debugging simpler than one URL that returns a different representation based on headers or user agent.)

## Every post gets a Markdown variant from the same content collection

The Markdown version is not a second copy I have to maintain manually. Astro reads the same content collection that builds the HTML page and generates a static endpoint from it.

Dutch posts use `src/pages/blog/[...slug].md.ts`. English and French posts follow the same pattern under `src/pages/[locale]/blog/[...slug].md.ts`. The endpoint adds frontmatter an agent needs to place the content:

```ts
const fmLines = [
  "---",
  `title: "${escape(post.data.title)}"`,
  `description: "${escape(post.data.description)}"`,
  `canonical: "${canonical}"`,
  `pubDate: "${post.data.pubDate.toISOString()}"`,
  post.data.updatedDate
    ? `updatedDate: "${post.data.updatedDate.toISOString()}"`
    : null,
  `category: "${post.data.category}"`,
  "---",
].filter(Boolean);

return new Response(fmLines.join("\n") + "\n" + post.body, {
  headers: {
    "Content-Type": "text/markdown; charset=utf-8",
    "Cache-Control": "public, max-age=3600",
  },
});
```

The important decision is `canonical`. The HTML page remains the public, canonical page. The Markdown route is an alternative format that points back to that page. I do not want two competing versions of a post, just two useful representations of the same source.

That is also why the URLs are language-aware. A Dutch post gets `/blog/slug.md`; an English post gets `/en/blog/slug.md`. The structure matches the page URL a human opens.

## The HTML page explicitly points to Markdown

A `.md` URL helps only when an agent can find it. The shared SEO component therefore places an alternate link in the `<head>` of every blog post:

```astro
<link
  rel="alternate"
  type="text/markdown"
  href={new URL(mdUrl, Astro.site).toString()}
/>
```

`BlogPost.astro` derives `mdUrl` from the post ID and language. A small detail matters here: Dutch content lives at the root, while English and French content have a locale prefix.

```ts
const mdUrl = id
  ? lang === DEFAULT_LOCALE
    ? `/blog/${id}.md`
    : `/${lang}/blog/${id.replace(`${lang}/`, "")}.md`
  : undefined;
```

Vercel's guide also covers `Accept: text/markdown`, a `Vary: Accept` header, and automatic agent detection. Those are useful when you want one URL to dynamically serve HTML or Markdown. I did not want cache variants, user-agent rules, or hidden switching here. An explicit `.md` route is boring, directly openable, and testable.

## Root `/llms.txt` needed to know the full collection

The AI SDK documentation itself uses [`llms.txt` as a Markdown entry point](https://ai-sdk.dev/docs/introduction) for tools including Cursor, Windsurf, Copilot, and Claude. That is the useful application for me: a small index that lets an agent see which content exists before it starts fetching individual pages.

The first version of my root `/llms.txt` had a problem: it included Dutch posts only. That is incomplete on a site with English and French versions. The branch therefore adds a collection-wide helper:

```ts
export async function getAllPosts() {
  return (await getCollection("blog")).filter(isVisible);
}

export const postMarkdownUrl = (post: BlogEntry) =>
  postUrl(post).replace(/\/$/, ".md");
```

The root index can then include every visible post, regardless of language, with a direct link to its Markdown version:

```ts
const posts = (await getAllPosts()).sort(...);

`- [${p.data.title}](${url(postMarkdownUrl(p))}): ${p.data.description}`
```

The existing `/en/llms.txt` and `/fr/llms.txt` routes remain compact language entry points. Root `/llms.txt` is the collection-wide map. Drafts do not accidentally end up there because `getAllPosts()` uses the same visibility rule as the production site.

For an agent looking for [my build log about a 24/7 assistant on a Raspberry Pi](/en/blog/openclaw-on-raspberry-pi/), that means less guessing: first the index, then the right Markdown source.

## The sitemap had the same multilingual gap

The Markdown routes already existed. So did the HTML alternate. The work in this branch was mostly about completing what looked correct in one language.

The sitemap code used to read only files directly under `src/content/blog/`. It could not set `lastmod` in the same way for posts in `en/` and `fr/`. This is a familiar collection bug: code works for the default directory until content moves into a subdirectory.

The fix now walks Markdown and MDX files recursively:

```ts
async function blogFiles(dir) {
  const files = [];
  for (const entry of await readdir(dir, { withFileTypes: true })) {
    const full = path.join(dir, entry.name);
    if (entry.isDirectory()) files.push(...(await blogFiles(full)));
    else if (!entry.name.startsWith("_") && /\.(md|mdx)$/.test(entry.name)) {
      files.push(full);
    }
  }
  return files;
}
```

For every post, the configuration prefers `updatedDate` and otherwise uses `pubDate`, then maps the file path to its public route. `/blog/.../`, `/en/blog/.../`, and `/fr/blog/.../` now use the same freshness rule.

That date is not a ranking button. It does tell a crawler or agent when a source last changed substantively. If I update [the practical lessons from three quantization rounds](/en/blog/quantization-local-llms/), every layer should expose the same signal.

## I test the output, not only the source code

The most important part is not a route, but `scripts/check-content.mjs`.

The check runs against `dist/`, so against what Astro actually built. It finds every page with `BlogPosting` JSON-LD and then checks whether that page has exactly one Markdown alternate, whether the route is correct, whether the Markdown output exists, whether the URL appears in root `/llms.txt`, and whether the sitemap date matches the frontmatter.

The core looks like this:

```ts
if (markdownAlternates.length !== 1) {
  err(`${page}: ${markdownAlternates.length} markdown-alternates (expected 1)`);
}

if (!linkResolves(markdownPath)) {
  err(`${page}: markdown endpoint missing at ${markdownPath}`);
}

if (!rootLlms.includes(`https://djangodevreng.nl${markdownPath}`)) {
  err(`${page}: missing from root /llms.txt`);
}
```

That is the boundary I care about. Code review can confirm that a route appears logical. Only built output shows whether the `<link>` really landed in the HTML, whether the static `.md` file exists, and whether the sitemap got the correct date.

Google recommends making important pages findable through contextual internal links and keeping link text descriptive. Its [guidance on crawlable links](https://developers.google.com/search/docs/crawling-indexing/links-crawlable) is not agent-specific, but the discipline is the same: a link should have a real URL and make clear where it goes. That is why this post does not sit apart from the rest of the blog, and why it links to the underlying build logs where that helps.

## What this implementation does not do

This is not a full agent-readiness specification.

I did not build `sitemap.md`, Markdown 404s, an MCP server, or automatic rewriting based on an agent user-agent. I also have not measured how many agents use the new routes or whether this changes visibility in AI answers. That would be a next measurement, not a claim I can make now.

The step that exists today is smaller and more useful: every published post has an explicit machine-readable representation, the index knows every language, freshness travels with it, and the build fails when those contracts break.

If I built this again, I would start in exactly that order. One source for HTML and Markdown. Then canonicals and dates. Then an index. Only then extra layers such as content negotiation or MCP, when a concrete agent use case requires them.