Prevent specific user from accessing sharepoint document libraries (also in Office applications)

[Kiril]

I managed to block access to SharePoint Online using a Conditional Access Policy, however when the user opens Word or Excel he can still access SharePoint sites and their respective document libraries.

[#2]

Did you already review this article? https://support.microsoft.com/en-us/office/customize-permissions-for-a-sharepoint-list-or-library-02...

[#3]

Yes, I did. The SharePoint Libraries in question are all public to our organization. There is no option to explicitly block access.