The LLM Latency Guidebook: Optimizing Response Times for GenAI Applications

I love the clarity of your guidebook and the concrete numbers around latency improvement. We've been working on making semantic caching work in conversational AI applications where the conversational context is important. By conversational AI, I mean more like your RAG example where someone is troubleshooting a product in an ongoing chat. The same question at one point in a conversation can have a different meaning at another point in the conversation. I'd love if you checked out how we're building context-awareness into our cache. I've been writing about it here: https://www.linkedin.com/company/canonical-ai