Markdown Chunker
Split long Markdown into smaller chunks by headings, tokens, or characters. Preserve document structure, add overlap, and prepare content for AI, LLM, and RAG workflows directly in your browser.
Processed locally in your browser — Markdown content is not uploaded to a MarkForge chunking server.
Release to load file
Accepted uploads: .md, .markdown, or .txt, up to 5 MB. Drag a file onto the editor, or use Upload File.
Need a more detailed token count? AI Token Counter →
Chunking Settings
A new chunk begins at each qualifying heading. Lower-level headings stay inside that chunk. Structure mode doesn't use a token/character target.
Target size is approximate when preserving complete Markdown structures.
Target size is approximate when preserving complete Markdown structures.
Overlap reuses complete trailing blocks from the previous chunk — never a raw character cut — so it approximates the requested amount without breaking Markdown structure. Disabled in Structure / Headings mode.
Preserve Heading Context
Prepend relevant ancestor headings to chunks that start deep inside the document's structure.
Generating chunks…
No chunks yet
Want to inspect how a chunk renders?
Preview Markdown →Need to tidy the source first?
Clean Markdown →Chunks
Copy All Chunks joins every chunk with a plain <!-- MARKFORGE_CHUNK_BREAK --> comment so you can split them apart later. This delimiter is never added to individual chunk files or the ZIP.
How to Split Markdown into Chunks
- Paste Markdown into the editor, or use Upload File (or drag and drop) to load a .md, .markdown, or .txt file.
- Choose a chunking method: Structure / Headings, Target Tokens, or Target Characters.
- Set a target size and, for the token and character methods, an optional overlap.
- Select Generate Chunks.
- Copy or download individual chunks, or use Copy All Chunks / Download All .zip for every chunk at once.
What Is Markdown Chunking?
Markdown chunking means splitting a longer Markdown document into smaller sections, or chunks, while trying to preserve useful document structure — such as headings, fenced code blocks, and tables — instead of cutting arbitrarily at a fixed character count.
People split Markdown for a range of reasons: preparing long AI prompts, organizing a large Markdown file into smaller sections, feeding content into a retrieval workflow, or simply processing a document in smaller, more manageable units. MarkForge's chunker runs entirely in your browser: it parses your Markdown's structure locally, slices the original source into non-overlapping blocks, and reassembles those blocks into chunks sized to a token or character target — or split around your document's headings.
Choose a Markdown Chunking Method
MarkForge works as a Markdown splitter for breaking large Markdown documents into smaller, ordered chunks while preserving useful structure. It slices chunks directly from your original source rather than regenerating Markdown from parsed tokens, so heading syntax, list markers, code fences, links, and whitespace stay exactly as you wrote them. Pick whichever of the three methods below best fits your document and how you plan to use the output.
By Headings
Chunks are created around qualifying headings in your document, so boundaries follow your Markdown's existing outline instead of a size target. This works well when your document's headings already reflect its logical sections. With Preserve Heading Context on, relevant ancestor headings can be prepended to chunks that start deep inside the structure.
By Tokens
Chunks are assembled up to an approximate token target, using the General AI Estimate or an exact local OpenAI o200k_base / cl100k_base encoding. MarkForge keeps whole Markdown blocks together rather than splitting them mid-block, so a protected block larger than your target stays intact in its own oversized chunk instead of being split unsafely.
By Characters
Chunks target a character count instead of a token count, which is useful when you want a simple, deterministic size limit. Keep in mind a character count isn't the same as a token count — the two will differ depending on the text.
Markdown Chunking for AI and RAG Workflows
Long documents are often split into smaller units before being used in AI prompts, LLM ingestion, or retrieval-augmented generation (RAG) workflows — an approach where a system retrieves relevant sections of a document, rather than the whole source, to help answer a query or provide context to a model. Markdown's own structure — headings, lists, tables, code fences, and other block-level elements — can provide useful boundaries to preserve while splitting, rather than cutting at an arbitrary character count.
MarkForge can help create structure-aware Markdown chunks using headings, token targets, or character targets, making the output easier to feed into a prompt, an embedding pipeline, or a retrieval index elsewhere. MarkForge itself only produces the chunk files locally: it does not upload your content, generate embeddings, connect to a vector database, or perform retrieval on your behalf.
Chunk Size and Overlap
Chunk size controls the approximate maximum amount of content in each section — measured in tokens, characters, or by heading boundaries, depending on the strategy you select.
Chunk overlap intentionally repeats some trailing content from the end of one chunk at the start of the next, so neighboring chunks retain a little shared context. MarkForge builds overlap from complete Markdown blocks rather than an arbitrary character cut, so it stays valid Markdown.
More overlap means more duplicated text across your chunk set. A larger overlap can help continuity between chunks, but it also increases the total amount of content you're storing or embedding, and there's no single overlap value that's right for every document.
Protected Markdown Structures
MarkForge prioritizes keeping fenced code blocks, Markdown tables, and YAML front matter intact as complete blocks, even when doing so causes an individual chunk to exceed your selected target size. Splitting through the middle of a code block or a table would produce invalid or misleading Markdown, which is worse than an occasionally oversized chunk.
When a single protected block is larger than your target on its own, MarkForge keeps it whole in its own chunk and marks that chunk with a small "Protected block exceeds target size" warning instead of corrupting the structure.
When your document begins with YAML front matter, MarkForge keeps it intact in the first chunk only — it isn't duplicated into later chunks or pulled into chunk overlap.
Prepare Markdown Before and After Chunking
Chunking works best on clean, well-structured Markdown. If your source has inconsistent formatting, you can run it through the Markdown Cleaner first, then check how it renders with the Markdown Previewer before splitting it.
After generating chunks, the AI Token Counter can help you inspect a chunk's token size in more detail or compare it against a specific model's context window. These steps are optional — you can paste Markdown directly into the chunker any time you don't need them.
Private by Design
Markdown parsing, chunk generation, token counting, and ZIP file creation all happen locally in your browser. Pasted Markdown, uploaded files, filenames, generated chunks, and token counts are not uploaded to a MarkForge server or any third-party API for chunking.
For additional details, see the MarkForge Privacy Policy.
Related Tools
PDF to Markdown Converter →
Convert PDF files into clean, structured Markdown directly in your browser.
DOCX to Markdown Converter →
Convert Microsoft Word DOCX files into clean Markdown directly in your browser.
HTML to Markdown Converter →
Convert HTML source or HTML files into clean Markdown directly in your browser.
AI Token Counter →
Count tokens, words, and characters and compare text against an AI context window.
Markdown Previewer →
Write or paste Markdown and see a live formatted preview in your browser.
Markdown Cleaner →
Clean and normalize Markdown formatting while preserving document structure.
Frequently Asked Questions
What is Markdown chunking?
Markdown chunking is the process of splitting a large Markdown document into smaller sections, or chunks, while trying to preserve useful structure such as headings, code blocks, and tables instead of cutting arbitrarily.
How do I split Markdown into smaller chunks?
Paste or upload your Markdown, choose a chunking method (headings, tokens, or characters), set a target size and optional overlap if relevant, then select Generate Chunks. From there you can copy or download individual chunks, or use Copy All Chunks / Download All .zip for every chunk at once.
Why split Markdown into chunks?
Splitting Markdown into smaller sections is commonly used to prepare content for AI prompts, RAG pipelines, vector databases, knowledge bases, document search, embedding workflows, and other LLM ingestion tasks where processing the entire document at once isn't practical.
What chunk size should I use for AI?
There is no single universal ideal size. The right chunk size depends on the downstream model's context window, how the chunks will be used, and the structure of your document. The AI Prompt preset uses a larger target (1,500 tokens) suited to sending a few chunks directly into a prompt; adjust the target to fit your specific use case.
What chunk size should I use for RAG?
RAG systems often use smaller chunks with some overlap so each retrieved section stays focused and self-contained. The RAG / Knowledge Base preset is a starting point (800 tokens, 100-token overlap), but the appropriate size depends on your retrieval system, embedding model, and document structure — there is no single ideal value.
Can I use Markdown chunks for RAG?
Yes. Chunks generated by MarkForge can be used as inputs to a downstream retrieval or RAG pipeline — for example, indexing or embedding them elsewhere. MarkForge itself only generates the chunk files locally; it does not perform retrieval or generate embeddings.
Does MarkForge use semantic chunking?
No. MarkForge currently chunks Markdown using document headings, token targets, or character targets, and can preserve Markdown structure and heading context along the way. It does not generate embeddings or calculate semantic similarity to choose chunk boundaries. To learn what semantic chunking is and how it differs from structure- and size-based methods, see What Is Semantic Chunking?
What is chunk overlap?
Chunk overlap intentionally repeats some trailing content from the end of one chunk at the start of the next chunk, using complete Markdown blocks, so neighboring chunks retain some shared context. More overlap means more duplicated text across your chunk set.
Does MarkForge split code blocks?
No. Fenced code blocks are treated as a protected structure and are never split merely to reach a target size. If a single code block is larger than your target, it stays intact in its own chunk, which is marked with a size warning.
Does MarkForge split Markdown tables?
No. Markdown tables are treated as a protected structure and are never split through the middle merely to reach a target size, for the same reason as code blocks.
What does Preserve Heading Context do?
When a chunk starts in the middle of a section rather than at a heading, Preserve Heading Context prepends the relevant ancestor and current headings — using their original Markdown syntax — so the chunk still makes sense on its own. It never duplicates a heading that's already at the start of the chunk, and it never duplicates YAML front matter.
Can I split Markdown by headings?
Yes. Choose the Structure / Headings strategy and select a heading level (Any, or H1–H6). A new chunk begins at each qualifying heading; lower-level headings stay inside that chunk.
Can I split Markdown by token count?
Yes. Choose the Target Tokens strategy, pick a token target, and choose a counting method: the General AI Estimate, or an exact local OpenAI o200k_base or cl100k_base encoding.
Are token counts exact?
The General AI Estimate is a rough, model-neutral planning figure based on text length. The OpenAI o200k_base and cl100k_base modes tokenize your exact chunk text using the selected local encoding. Either way, the target size is treated as approximate whenever preserving complete Markdown structure takes priority.
Can I upload a .md file?
Yes. MarkForge accepts .md, .markdown, and .txt files up to 5 MB, read entirely in your browser. You can also drag and drop a file onto the editor.
Can I download all chunks at once?
Yes. Download All .zip builds a ZIP file containing one .md file per chunk, generated locally in your browser. You can also use Copy All Chunks to copy every chunk at once, separated by a plain Markdown comment delimiter.
Does MarkForge upload my Markdown?
No. Pasted Markdown, uploaded files, generated chunks, and token counts are processed locally in your browser and are not sent to a MarkForge server or any third-party API.
Is the Markdown Chunker free?
Yes. The Markdown Chunker is free to use and does not require an account.