Quickstart

Chunk your first document in three lines.

from blazechunk import TokenChunker

chunker = TokenChunker(chunk_size=512, chunk_overlap=64)

# sync
for c in chunker.chunk("... a long document ..."):
    print(c.text, c.start_index, c.end_index, c.token_count)

# many documents
batches = chunker.chunk_batch(["doc one ...", "doc two ..."])

Tip

Every chunker in blazechunk works exactly like this — swap RecursiveChunker for any other chunker and the four methods (chunk, chunk_async, chunk_batch, chunk_batch_async) are identical.