A successful embedding request does not establish that every sentence in a long input was represented. Ollama’s embed API has a truncate option, which defaults to true. It can shorten an input that exceeds the model context. When an answer near the end of a long document never appears in search, inspect the chunk sent to the embedding endpoint before changing the chat model.
Test a chunk with an answer near the end
Make a synthetic document with a unique fact near its beginning and a second unique fact near its end. Use fictional values so you can safely record the test. For example, place a project’s approval code in the final section, then query for that code’s purpose.
Record the exact input chunk, model identifier, Ollama version and request options. Query with the same embedding model used for the index. Compare a short chunk containing the final section with the whole long input. If the short chunk retrieves well and the long one does not, you have a useful direction to investigate. This comparison is an illustrative test, not a measured benchmark.
Reject overlong input during diagnosis
During ingestion tests, send truncate set to false so an input exceeding the supported context produces an error you can handle. Treat a failed chunk as unfinished ingestion and retain its document ID for retry. Confirm the behavior on your installed Ollama and model combination, especially if a recent update changed long-input handling.
Choose chunk boundaries that preserve a useful unit of meaning, such as a section with its heading. Overlap can keep a sentence at a boundary understandable, but copied context also consumes space. Tokens and characters differ, so a character count alone is a rough estimate of whether a chunk will fit. Test representative languages and document formats.
Verify retrieval before judging the answer
Re-index the test document after changing the chunker and inspect the returned passage IDs and text. Confirm that the end-of-document fact is actually retrieved. Then pass those passages to the answer stage and check the final answer against the original.
Log chunk count and failures per document. If a document produces eight planned chunks but only seven are indexed, a search result can look plausible while missing the only relevant evidence. Keep the original document and an ingestion version so you can explain which preprocessing created a result. Roll out the revised chunker to a separate index first, and compare a small set of known-answer queries before switching.