One File or Many? Splitting a Book-Length Markdown Document

Keep a Markdown document in one file while it is short, and split it into one file per chapter once it gets long, shared, or hard to reorder. Splitting costs little, because Pandoc joins the files back into a single book when you export.

This guide is for writers, students, and developers with a long Markdown project: a thesis, a manual, a book. It covers when to split, how to name and order the chapter files, how to combine them on export, and what changes for metadata, links, and footnotes once your text lives in several files.

When one file is still the better answer

A single file is simpler for anything short. Search works across the whole text, one outline covers it, and nothing can fall out of order. If navigation is your only worry, headings that stay navigable solve it without splitting anything.

Splitting starts to pay off when one of these is true:

• The file is slow to scroll and you keep losing your place.
• More than one person edits it, so changes collide in the same file.
• You reorder chapters often, and cut-and-paste of huge blocks is risky.
• You want to review or commit one chapter at a time. Once you version-control your Markdown with Git, a diff for a single chapter is far easier to read than one for a whole book.

There's no exact word count where this flips. It's a judgment call, but if you're asking the question, you're probably close.

Name and order the chapter files

Order is the one thing you have to design up front. The phd_thesis_markdown template compiles chapters in alphabetical order and suggests numeric prefixes such as 01_chapter_1.md and 02_chapter_2.md. Do the same, with a short descriptive name after the number:

01-introduction.md
02-background.md
03-method.md
04-results.md
05-conclusion.md

A practical tip: number in steps of ten (10-, 20-, 30-) so you can insert a chapter between two others without renaming everything. Use two digits at least, or 10- will sort before 2-.

Two habits keep the joins clean:

• End every file with a blank line. The template's README warns that a chapter must finish with at least one blank line, or the next chapter's heading may not be recognised.
• Start each file with its own top heading. Pick one heading level for chapters (H1 is the usual choice in a book) and use it in every file.

Combine the chapters on export

Pandoc does the joining. Its manual says it will concatenate all the input files, with blank lines between them, before parsing, so the result is parsed as one document:

pandoc 01-introduction.md 02-background.md 03-method.md \
  04-results.md 05-conclusion.md -o book.pdf

The order you give is the order you get. A shell glob such as pandoc [0-9]*.md -o book.pdf expands alphabetically, which is why the numeric prefixes matter.

Once the command gets long, move it into a defaults file. Pandoc's input-files list accepts several Markdown files, and --defaults loads the whole thing:

# book.yaml
input-files:
  - 01-introduction.md
  - 02-background.md
  - 03-method.md
  - 04-results.md
  - 05-conclusion.md
toc: true
number-sections: true
output-file: book.pdf

Run it with pandoc --defaults book.yaml. The chapter order now lives in a versioned file instead of your shell history. For LaTeX output you can add top-level-division: chapter, which the manual says is implied when the document class is report, book, or memoir. For more on formats and flags, see exporting Markdown to PDF and Word with Pandoc.

Keep shared metadata in one place

Title, author, and bibliography settings belong to the whole book, not to a chapter. Put them in a single metadata.yaml and pass it with --metadata-file:

---
title: "A Field Guide to Plain Text"
author: "Your Name"
bibliography: references.bib
---

The manual notes that --metadata-file can be repeated and that values from files given later win over earlier ones. Leave YAML front matter out of the chapter files themselves, so there is only one place to edit the title.

Links, images, and footnotes across files

By default, Pandoc parses the concatenated text as one document. A link to a heading in another chapter, or a footnote defined in a different file, resolves as though everything were in one file.

The --file-scope option changes that. It parses each file separately, which helps if two chapters reuse the same footnote label, but the manual is blunt about the cost: footnotes and links will not work across files. Identifiers also gain file-name prefixes, so a header foo in subdir/file1.txt becomes subdir__file1.txt__foo. Leave it off unless you hit label clashes. See footnotes in Pandoc for the syntax.

Two more things to watch:

• Relative links between chapter files work when you browse the repository on GitHub, which resolves them from the current file's location. They point at files, though, not at headings in the combined book, so check them in the exported output too.
• Images are found relative to the working directory by default. If you set --resource-path, list the working directory explicitly, or Pandoc stops searching it.

A folder layout that scales

The thesis template separates content from output: source/ for chapters and the bibliography, style/ for styling files, output/ for the finished build, and a figures/ folder inside source/ for images. A similar tree for a book:

book/
  book.yaml
  metadata.yaml
  source/
    10-introduction.md
    20-background.md
    figures/
    references.bib
  output/

You edit only what's under source/. Pandoc writes to output/, which you can keep out of version control.

Doing this with Carets

Carets is a native notes and code editor for iPhone, iPad, and Mac, and a folder of numbered chapter files is the kind of project it handles well. It edits Markdown and plain text directly, with syntax highlighting that makes headings and the YAML in metadata.yaml and book.yaml easy to scan. Files, projects, and tags let you keep the chapters, the metadata file, and the defaults file together as one project, and everything stays plain text, portable and future-proof.

That means you can fix a paragraph in chapter three on your phone and run the Pandoc export later on your Mac, without a conversion step in between. Start with two files, check the export, then split the rest.

Download Carets from the App Store for iPhone, iPad, and Mac.

Frequently Asked Questions

Should I split a Markdown book into one file per chapter?

Usually yes once the draft passes a few thousand words or has several authors: smaller files diff cleanly, open fast, and are easy to reorder. A short document is easier to keep in one file.

How does Pandoc know the order of my chapter files?

Pandoc uses the order the files are given on the command line or in the input-files list of a defaults file. A shell glob sorts alphabetically, so numeric prefixes such as 01- and 02- keep reading order and sort order the same.

Why does the first heading of a chapter not render after combining?

Pandoc inserts blank lines between files, but tools that join files by plain concatenation may not. End every chapter with a blank line so the next file's heading starts on its own line.

Do links between chapter files work in the exported book?

Links to headings work when Pandoc parses everything as one document. With --file-scope, links and footnotes do not work across files, and identifiers get file-name prefixes.