Forum Discussion

vaibhav1080's avatar
vaibhav1080
Copper Contributor
May 27, 2024

How to retrieve all the documents on SharePoint using custom copilot.

Hi, I have created a custom Copilot on a SharePoint site for retrieving all documents stored on the SharePoint site. However, in response, I don't get all the relevant documents; I only get the top 2-3 documents. Is there any way to retrieve them all at once based on a user query?

  • Sajjad's avatar
    Sajjad
    Brass Contributor

    How to retrieve all the documents on SharePoint using custom copilot.
    Hi, I have created a custom Copilot on a SharePoint site for retrieving all documents stored on the SharePoint site. However, in response, I don't get all the relevant documents; I only get the top 2-3 documents. Is there any way to retrieve them all at once based on a user query?
    Show drafts

    Retrieving all documents on SharePoint using a custom Copilot can be achieved, but the default settings might only show a limited number. Here's how to potentially get all documents:

    Check Search Scope:
    Ensure your Copilot is configured to search across the entire SharePoint site collection, not just a specific library or folder.
    In Copilot Studio, navigate to the "Content Sources" section and verify it includes all relevant libraries and folders.
    Modify Search Parameters:
    Within Copilot Studio, explore options for modifying the search query. You might be able to adjust parameters like "row count" or "paging" to retrieve all results.
    Consider Pagination:
    If modifying search parameters isn't feasible, explore a two-step approach:
    In the initial query, instruct Copilot to return a high number of results (e.g., 100).
    Then, within your Copilot response, guide users on how to refine or filter the extensive list based on their specific needs.
    Additional Tips:

    Test Thoroughly: After making any changes to Copilot's configuration, test it thoroughly with various queries to ensure it retrieves all documents as expected.
    Performance Considerations: Retrieving a massive number of documents might impact performance. Balance the need for completeness with user experience.
    Alternative Approach (For Informational Documents):

    If your Copilot primarily deals with informational documents, consider leveraging SharePoint Search instead:

    Utilize SharePoint's built-in search functionality to allow users to find relevant documents based on keywords or filters.
    While Copilot excels in contextual interactions within documents, SharePoint Search might be better suited for comprehensive document retrieval across the site.
  • frankfalvey's avatar
    frankfalvey
    Iron Contributor
    To retrieve all the documents on SharePoint using a custom Copilot, you can follow these steps:
    Create a Custom Copilot:
    You can create a custom Copilot from SharePoint that’s grounded in the curated, authoritative content you choose. This can be done with just a few clicks, whether you’re an admin or a business user.
    Enable Copilot for SharePoint Site:
    Site owners can enable any site to get a built-in Copilot, scoped to the content of that site. This Copilot can interact with the content on the site and can be shared to other Microsoft 365 apps and experiences.
    Customize Your Copilot:
    If the default scope doesn’t work for your site, you can customize the Copilot by changing the content, identity, and behavior. You can also choose a Copilot that you’ve created through Microsoft Copilot Studio or Visual Studio to be the default Copilot for the site.
    Deploy Copilot to SharePoint:
    You can deploy a custom Copilot to SharePoint with a seamless single sign-on experience. This involves setting up app registration configurations required for SSO, setting up a Generative Answers node for your Copilot over a SharePoint or OneDrive data source, and building the SPFx component from scratch to bind it to your Copilot.
    Use Generative Answers:
    On the Create generative answers node, select Edit under Data sources. Add the SharePoint or OneDrive for Business URLs you want to use to allow the Copilot to access and retrieve documents based on the site’s pages and documents.
    By following these steps, you can effectively use a custom Copilot to retrieve and interact with documents on SharePoint, enhancing productivity and collaboration within your organization.
    • vaibhav1080's avatar
      vaibhav1080
      Copper Contributor
      Hello frankfalvey, I have followed all the steps you mentioned. I am testing the bot, but it is not retrieving all the files. I would like to access the entire folder content when I specify the folder name. not just top 2-3 results.
      • frankfalvey's avatar
        frankfalvey
        Iron Contributor
        If your custom Copilot is not retrieving all the files on SharePoint, here are some troubleshooting steps you can try below.

        Check Permissions:
        Ensure that the Copilot has the necessary permissions to read files from SharePoint. The required scopes are Sites.Read.All and Files.Read.All. These should be specified in the Microsoft Entra ID App Registration.
        File Size Limit:
        There is a file size limit for the files that Copilot can process. Files larger than 3 MB may not be retrieved. If you have larger files, consider splitting them into smaller files.
        Update Authentication Settings:
        Verify that the authentication settings configured in the Copilot are correct. The Copilot must use manual authentication with the Service Provider type of Microsoft Entra ID (formerly Azure Active Directory).
        Refresh Data Sources:
        If you’ve recently added new files to SharePoint, make sure to refresh the data sources in the Copilot Studio so that the Copilot can access the latest content.
        Check File Types and Names:
        Ensure that the filenames and file types are supported by SharePoint and that they do not contain any special characters or formats that are not recognized.
        Sync Issues:
        If there are sync issues with SharePoint, consider updating to the latest version of OneDrive, stopping syncing and running setup again, or unlinking OneDrive and running the setup again.
        By following these steps, you should be able to resolve the issue with your custom Copilot not retrieving all files on SharePoint.
  • KaPat153's avatar
    KaPat153
    Copper Contributor
    To retrieve all documents from a SharePoint site using a custom Copilot, ensure that your Copilot is configured to search across the entire SharePoint site, not just limited to a few documents.
    • vaibhav1080's avatar
      vaibhav1080
      Copper Contributor
      Hello @KatPat153, I appreciate your response. My use case is limited to a single SharePoint site, which I have included in the knowledge base, but I am unable to access all the content within a particular folder. Also, is there a way to incorporate various file types, such as .PSD or JPEG, that are not typically part of the Microsoft ecosystem?

Resources