As far as I have been able to find you are correct, there is no method of removing the Forms button from the Teams app.
This can be reversed by deleting the Form and then the associated columns from the SharePoint document library, however this isn't the issue.
Microsoft have introduced a button to the Teams app that is for a SharePoint feature, in the vast majority of cases in our environment it is not appropriate to create a SharePoint Form in a Team due to the metadata column it creates in the DL, these will not be appropriate for many documents in a generally Team. People see a button called "Forms" and, understandably, click it expecting to create a Microsoft Form, as it's a Microsoft application they already use, they then are confused as to why they now see additional columns for all their files under the Shared tab, and why they now have warnings messages.
This causes support call's, and requires time to reverse what's been done.
There should a Team Policy setting to control the visibility of this Forms button in the Teams App, and/or to disable the feature if required.
It is confusing named the same as an existing service and so people think they are using that, it is not appropriate for general Teams especially if most people are only using Teams and not SharePoint.
The feature is useful for the specific scenario's it's created for, but it is not for the vast majority of our Teams and usage and is causing confusion and draining support time, there should be more options for controlling it's usage.