Microsoft
Microsoft
Microsoft
>>1) I am noticing that sometimes the response includes the citation with a link to the source content and it is also stated in the response but the actual superscript and links at the bottom are missing. Is this a bug?
Maybe the generated reference had the wrong format? If you have an example (both UX and the output from "Thought process") I'm happy to take a look.
>> 2) I viewed the "though process" for one of our question. The "Sources" included 3 chunks of data and it's total was roughly 606 tokens. Do "Sources" contribute to the token calculation? The sources might exceed 4000 tokens. How can we avoid this?
Yes, sources contribute to the token count since they end up being part of the prompt. The retriever is good enough to find candidates but ultimately we need GPT to tell which are usable documents, so until we send it to GPT we don't know which ones we need, which means we do need to use a few of the documents. We chunk documents to a relatively small size to minimize the chances of exceeding the context lenght limit.