Cache large documents and contexts on Google's servers so you can ask multiple questions without re-uploading or re-processing them — saving both cost and time. This project implements the context ...

LLM API costs scale with every request. In any real application, a significant fraction of prompts are semantically identical — same question, different phrasing. Standard caches miss all of them ...