"Edit in grid view" for selected people

Meow13  Take a look at this guide

https://tayfuntech.com/2021/08/15/edit-grid-view-option-disappears-in-sharepoint/

It's for granting access to grid view for external users, but you may be able to figure out a way to use the same premise to configure permissions for groups on the library to your liking.

If you want to add or remove a single permission to a role (for example, the UI automatically checks off others when you select EditListItems permission), you can do that using Add-PnPRoleDefinition or Set-PnPRoleDefinition in the PowerShell PnP library.

*edit*

My thinking is that you could this by creating folders in the library to share with this group (let's call it "No Grid Group" for now) to work out of. Then you would break the library inheritance, remove the EditListItems permission from "No Grid Group" by giving that group a custom Role without EditListItems at the library level, and then you'd share the folders with "No Grid Group" for them to work out of and be unable to use grid view.

I haven't tested it myself, and my example may require some tweaking, but you should be able to get it working since I have tested it with external users like the guide shows.

Unfortunately, there's no simple switch you can flip for this.