ErikEjlskovJensen confirmed w/ the PG that currently only way supported is to create the identity in the database. The next post in this series will involve the security seeding scripts for Entra Groups/members via environment specific post scripts. The TL/DR is the user will need to be created via SQL within the CI/CD Process. Outside of this the SqlAzureDacpacDeployment@1 - Azure SQL Database deployment v1 task | Microsoft Learn can run a script file or inline to create the user. One could also achieve this via sqlcmd commands.
The only requirements will be that the SQL Server has an identity associated to it which can read Entra to validate the identity. One call out though is the SQL DB will store the Entra ID thumbprint....so if one were to drop and recreate the App Service's System Assigned Identity or User Assigned Identity associated with it they'd have to do the same on the SQL DB to store the new thumbprint.