Concerning Consistent Hallucinations

[agreca]

Every once in a while when I try to have BCE provide a page summary for a PDF using the Edge Bar, it'll come up with information about a Korean Room-temperature superconductor (LK-99). I have never searched about this and I'm not sure why it is coming up. At first I thought it was a fluke and that it misinterpreted some information from the document but I've had at least 3 occurrences now..

Another instance below shows that it was partially correct when returning info from the page but then in the same response, it lists the URL to the phys.org page talking about the superconductor. I was actually able to reproduce the issue twice with this same document after closing Edge completely. (identifying info has been redacted)

Edit: I should also mention that the document doesn't contain anything pertaining to this superconductor or any references to the article mentioned.

[#2]

@agreca Experiencing the exact same issue. 

[#3]

Thank you for bringing this to our attention. We are currently investigating this.

[#4]

Hi @agreca and @MikevZ because commercial data protection is applied and chat data isn't stored, we're unable to see exactly what might be triggering Copilot to hallucinate in this way. If you're using Creative mode in Copilot, hallucinations are more likely. We recommend that you switch to Balanced mode in Copilot when asking for a page summarization to get a more accurate response.

[#5]

Hi @Eric_VanAelstyn, I almost always use Precise and this was the case when I received these results.

[#6]

Hi @Eric_VanAelstyn , I am able to almost consistently reproduce this issue with a Statement of Work document that has no mention of this LK-99 superconductor. What's strange is it happens when Copilot is trying to build a reference to info it finds in the document. See below..