ADF Adds Support for Inline Datasets and Common Data Model to Data Flows

Quentin_Fisher I had bit longer session with support regarding MS providing a work around for this. While they were helpful and appreciated their support, it's pretty much up to us to make the scripts to work around. We first tried to use databricks and python to handle but was too cumbersome for such simple tasks/actions. I'm personally more proficient with PowerShell so my plan was to use a pipeline to trigger an Azure Automation PowerShell script to download the json file, edit it in memory and reupload it to datalake gen2 storage. Then in the same script update the file name path.

I got partially there but ran into the issue that I couldn't just say, replace : with X  for this string, as that would edit the https: in the string. I was going to split it by / and work some logic to fix it up but never got around to it. I'll share what I made so far, I don't remember if it even works in the state my code is in now, but maybe someone can finish it. I unfortunately don't have the time at the moment to fully flush it out. But I'll probably start tackling it again in a month or two. Who knows maybe MS will fix it by then!

Connect-AzAccount
Select-AzSubscription -SubscriptionId 00000000-0000-0000-0000-0000000000
$ctx = New-AzStorageContext -StorageAccountName 'StorageAccountName' -UseConnectedAccount

$filesystemName = "power-platform-dataflows"
$filePath = "environments/orgID/dataflowname/model.json"
$dirname = "environments/orgID/dataflowname/"

$downloadFilePath = "C:\Temp\model.json"
$localSrcFile =  "upload.txt"
$destPath = $dirname + (Get-Item $localSrcFile).Name


Get-AzDataLakeGen2ItemContent -Context $ctx -FileSystem $filesystemName -Path $filePath -Destination $downloadFilePath

Get-AzDataLakeGen2ChildItem -Context $ctx -FileSystem $filesystemName -Path $dirname -OutputUserPrincipalName

New-AzDataLakeGen2Item -Context $ctx -FileSystem $filesystemName -Path $destPath -Source $localSrcFile -Force

Move-AzDataLakeGen2Item -Context $ctx -FileSystem $filesystemName -Path $dirname1 -DestFileSystem $filesystemName -DestPath $dirname2


$Contents.entities|Foreach {
    #$PathParts = $_.partitions.location -split '/'
    $_.partitions|Add-Member -NotePropertyName 'location' -NotePropertyValue ($_.partitions.location -replace '_', ':') -Force
    
}