JeremyTBradshaw it depends on what protocol you are using to send.
If you are using Exchange Online API, then SMTP protocol is used "underneath". SMTP protocol requires a user. So even for application running with client credential flow you must specify a user, which is basically impersonated by the application. Hence, you must set which mailboxes service principal can impersonate. Granting SMTP.SendAsApp is not enough. The documentation could be clearer, but so far it says "Allows the app to have send access to all mailboxes". It doesn't say "Allows the app to send as any mailbox" 🙂
Alternatively, if you are using MS Graph API, then you either:
a) set Mail.Send permission on API level and allow access to all mailboxes, or
b) don't set any API level permissions at all, but configure RBAC with "Application Mail.Send" role and set the role to apply to needed mailboxes.