Event banner
Event details
Please do not delete this; I haven't received an actual response from Microsoft yet.
I have been using 365 Chat, especially to ask questions about documents stored within a SharePoint to which I have access to. It does okay with general questions ("Summarize this document", "What are the main takeaways of this contract?"), but really struggles with more precise questions ("What is the agreed bonus on this contract?", "What are the special clauses pertaining to confidentiality in this document?"). Why is that, and do you plan on improving this? To give a comparison, using Azure OpenAI's Bring Your Own Data on the same documents works much better.
- Hi, We have found exactly the same. It doesnt help that Copilot never learns, it doesnt get more intelligent. You, the human get smarter at prompting giving the impression that Copilot is improving, when, actually its you. Do better prompts. Offer more context. Explain more. Always think first that the problem lies with you and not the tech, then address where you think you may have gone wrong, try again and you WILL get better results. But if you immediately assume the tech is flawed, you will never improve your prompting and never get better results. Remeber, it is a young, inexperienced intern that you need to guide and coach, but more importantly, you need to be very specific about what it is you want. Your prompt for instance "Wat are the special clauses pertaining to confidentialy in this document". If you ask your 12 year old that question would they understand? What do you mean by "special"? What is the arameter for Special? How would I know a clause is special? Are there certain characteristics that it would have that would indicate to me that it is special? How would I know if a clause Pertains to confidentiality? What type of confidentiality? In which context would the clause be pertainint to? In which circustance? What if this document has no confidentiality clauses? Pertain to what exactly - the clause or the document? The whole document? A better prompt might be: If you want a list of confidentiality clauses: "Please identify all clauses within this document that directly address confidentiality." If you wanted an explanation of specific confidentiality terms: "Could you please explain the confidentiality obligations outlined in clauses [list clause numbers]? How do they apply in [describe a relevant scenario]?" If you are concerned about the strength of the confidentiality provisions: "Do the confidentiality protections in this document meet industry standards? Are there any potential weaknesses or areas for improvement?" Additional Tips: Context is key: A brief explanation of why this analysis is needed would provide better focus. Specificity Wins: The more specific the question, the more tailored and helpful the answer will be. Let me know if works.
- Thanks for the tips. However, the problem seems to be a bit more deep-rooted than that: 365 Chat wouldn't even be able to look in the right document, unless I specifically reference that document in my question. Second, I've tried using very specific questions (Ex: "What are the amounts, in USD, of the possible bonuses detailed in the contrat between A and B, and to what sport do they apply?") and the answer would say that such information is not present inside the document. This happened even when referencing the specific document in which such info is present.
- Hey Nikitas, thanks for the feedback! Have you tried asking these questions in e.g. Copilot in Word to ask the question about the doc? I'd also recommend using the thumbs up/down button to provide feedback on the prompt experience so we can continue to improve!
- I will try again tomorrow, but today, as much as I tried in Word, I always got the error message similar to "Your request cannot be processed, please try again later" (for any type of query). More importantly, what if I want to ask a question about a PDF document? I couldn't open it in Word without messing up the format. And what if I'm not sure exactly where that document is, but I want an answer about it? Isn't 365 Chat supposed to be able to do that? Thanks for the help!
- Have you tried todo the same with the document open in the browser (PDF) with the Copilot inside of the Edge browser? Just if the experience for you is the same.
- I had the document open in the browser, but wasn't using Edge. Is that supposed to make a difference ?
