I have the registry setting UseNexusForGameBarEnabled set to 0.
The setting "Open Xbox Game Bar using Nexus button on a controller" is also unchecked and disabled in the Game Bar application. All shortcuts are default with nothing extra configured other than this being disabled.
Regardless of both of those being configured, the second I click the nexus button on a controller it opens up the Game Bar on top of whatever application I am running.
This is especially annoying when I am in another application that makes use of the Nexus button such as PS Remote Play, since game bar pops up on top of the remote play application intercepting my press of the Nexus button.
So it appears this setting is being completely ignored on my system.
In this specific instance the controller being used is a PS5 Dual Sense controller, and the button being pressed is the Playstation Button.