Since I also found it confusing, I asked Copilot. 
Here's the reply:
Custom Copilots in SharePoint and Copilot agents in SharePoint are related but serve slightly different purposes:
Custom Copilots: These are AI tools that users can create and tailor to specific needs within SharePoint. They can be customized to interact with content in a particular way, providing specialized assistance based on the curated content of a SharePoint site 1. For example, a marketing team might create a custom copilot to manage and provide insights on their campaign materials.
Copilot Agents: These are pre-configured AI assistants that can be quickly set up and shared within SharePoint. They help automate tasks and answer questions based on the content available in SharePoint. Copilot agents are designed to be easy to create and use, even for those without coding skills 2.
Both types of Copilots leverage AI to enhance productivity and collaboration, but custom Copilots offer more flexibility and personalization, while Copilot agents provide a more straightforward, out-of-the-box solution.