SuperRAG – How to achieve higher accuracy with Retrieval Augmented Generation

Hi Andrew,

Thank you for sharing this insightful post! I have a quick question that I hope you can help me with. You mentioned, "By using this prompt and GPT-3.5 to evaluate the documents, we can see that document 2 is much more relevant to answering the user’s question," and provided the JSON data. Could you clarify how the "confidence" value is obtained? Is it derived from the log probabilities in the softmax layer of the LLM? Since GPT-3.5 is a closed-source model, I was wondering how this value is accessed.

Thanks in advance for your help!

Best regards,
Haibo