Skip to content
Changelog

Knowledge fixes for DocumentChunking and the CSV readers

October 1, 20261 min read

Agno's DocumentChunking and both CSV readers each got a fix for input that went wrong without a clear error. The outputs below are real, from the previous release and this one.

DocumentChunking checks its overlap

FixedSizeChunking and RecursiveChunking already raised for overlap >= chunk_size. DocumentChunking accepted it and built chunks longer than chunk_size, so the three strategies disagreed about a valid setting. It now runs the same check in its constructor:

DocumentChunking(chunk_size=100, overlap=150).chunk(doc)
# Before: chunk lengths [109, 206, 206, 206]
# Now:    ValueError: Invalid parameters: overlap (150) must be less than chunk size (100).

CSV readers handle CR record endings

CSVReader and FieldLabeledCSVReader wrapped uploaded content in a StringIO with the default newline handling. For records that end in \r, csv.reader raised new-line character seen in unquoted field, and the readers logged it and returned no documents. Uploads in both sync and async mode hit this, and so did async reads of a file path. All six wrappers now pass newline="", as the sync path reads already did, which accepts LF, CRLF and CR and keeps quoted multiline cells. Reading name,city\rAlice,Paris\rBob,Rome\r:

Before: CSVReader 0 documents, FieldLabeledCSVReader 0 documents
Now:    CSVReader 3 documents: 'name, city', 'Alice, Paris', 'Bob, Rome'
        FieldLabeledCSVReader 2 documents: 'Name: Alice\nCity: Paris', 'Name: Bob\nCity: Rome'

See the cookbook, and learn more about document chunking and the CSV reader in the documentation.

Frequently asked questions

An overlap equal to or larger than chunk_size made every chunk grow past the limit. DocumentChunking now refuses that setting with a ValueError, the same check FixedSizeChunking and RecursiveChunking run.

If the file ends each record with a bare \r, Python's csv module raised new-line character seen in unquoted field and the reader returned []. CSVReader and FieldLabeledCSVReader now read LF, CRLF and CR endings.

Shipped around the same time