You should not rely on IP addresses, especially between two cloud services, it's "old school". We are in a mobile world, and at the very latest this pandemic stuff should teach us that work is done from other locations, not just the office - so location (or IP address) can't really be relied on.
This is what user access rights are for. Once the contractor leaves, revoke his user rights. You can also recycle the keys (you should recycle them anyway, periodically). You also need to audit the access regularly: who has access to where, and is it justified or should it be revoked.
Between App Service in Azure, and Azure DevOps, you can use a service connection (Service Principal or Managed Identity) to publish apps. These require rights to be created and to grant access, that no one outside your company has -> not "everyone in US east" has that access.
This is a big change in thinking and I know personally all too well, that it takes time for security people to come around and embrace the "new age". But it will be worth it in the end, so keep on educating them 🙂
