TableChunker

Split large Markdown/HTML tables into rows, repeating the header in every chunk.

Tip

When to use: Large data tables in Markdown or HTML where you need to preserve table structure and repeat headers for context.

Initialization

from blazechunk import TableChunker

chunker = TableChunker(
    tokenizer="row",           # size unit (always "row")
    chunk_size=10,             # rows per chunk (excluding header)
)

Parameters

Parameter	Type	Default	Description
tokenizer	str	"row"	Always row-based counting.
chunk_size	int	3	Number of rows (excluding header) per chunk.

Usage

from blazechunk import TableChunker

table = """| Name | Age |
|------|-----|
| Alice | 30 |
| Bob | 25 |
| Charlie | 35 |"""

chunker = TableChunker(chunk_size=1)
chunks = chunker.chunk(table)
for c in chunks:
    print(c.text)
    print("---")

Output

| Name | Age |
|------|-----|
| Alice | 30 |
---
| Name | Age |
|------|-----|
| Bob | 25 |
---
| Name | Age |
|------|-----|
| Charlie | 35 |
---

Batch processing

chunker.chunk_batch(tables)            # sync
await chunker.chunk_batch_async(tables) # async

Warning

TableChunker is the one documented exception to the slice invariant: each chunk repeats the table header, so chunk.text != original_text[start_index:end_index].