Microsoft
About the resources not deleted:
Since the ARM template deployment is an incremental operation, the resources are not deleted by design. You need to use a Post deployment PS script in order to do so, this script is included in the output of the package (PrePostDeploymentScript.ps1). https://docs.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment#script about this script and how to implement it in your release definition.
Regarding the Git problem:
It is possible that you are including global parameters in the ARM template. Since global parameters are located in the factory entity, when they are added, the factory entity is added in the ARM template as well, however, the Git configuration is not. You can check the box to do not include them in the ARM template and use the Global Parameters post deployment script in order to update them in the target environment. This script is provided in the output of the package as well (GlobalParametersUpdateScript.ps1). Here's more info about https://docs.microsoft.com/en-us/azure/data-factory/author-global-parameters#cicd, and here is more info about the https://docs.microsoft.com/en-us/azure/data-factory/author-global-parameters#deploying-using-powershell.
