ImageStore keeps copied package and provisioned packages.
Sequence of provision in Best Practice:
- Copy package to ImageStore with compress option
- Provision package
- Remove package in ImageStore
- Upgrade app/cluster
- Unprovision old version
When step 3 and 5 are missing, ImageStore accumulates files.
Symptom:
- The ImageStoreService (fabric:/System/ImageStoreService) could fill up disk.
- ImageStoreService replica may take long time in InBuild.
Option/Configuration for automatic cleanup:
How to configure cleaning up copied application package (automatic Step 3)
- Register-ServiceFabricApplicationType -ApplicationPackageCleanupPolicy Automatic
At step 2, Register (a.k.a. Provision), the application package is deleted after successfully registering the application type.
- <Section Name="Management">
<Parameter Name="CleanupApplicationPackageOnProvisionSuccess" Value="False" />
</Section>
This configuration enabled automatic cleanup of application package after successfully registering the application type.
How to configure cleaning up automatically unused application type (automatic Step 5)
<Section Name="Management">
<Parameter Name="CleanupUnusedApplicationTypes" Value="true" />
<Parameter Name="PeriodicCleanupUnusedApplicationTypes" Value="true" />
<Parameter Name="TriggerAppTypeCleanupOnProvisionSuccess" Value="true" />
<Parameter Name="MaxUnusedAppTypeVersionsToKeep" Value="3" />
</Section>
Manual Cleanup:
When ImageStoreService must be cleaned up manually, you can follow this steps.
1. Delete copied packages
#Delete Content from Image Store
$content =
Get-ServiceFabricImageStoreContent -RemoteRelativePath "\" -ImageStoreConnectionString fabric:ImageStore
foreach($folder in $content)
{
Write-Host Working on $folder.StoreRelativePath
if (($folder.StoreRelativePath -ne "Store") -and ($folder.StoreRelativePath -ne "WindowsFabricStore"))
{
Write-Host Deleting $folder.StoreRelativePath
Remove-ServiceFabricApplicationPackage -ApplicationPackagePathInImageStore $folder.StoreRelativePath -ImageStoreConnectionString fabric:ImageStore
}
}
2. Unregister
Unregister-ServiceFabricApplicationType will remove application packages from image store as well as ImageCache on nodes (after a while).
Scenario1 : Seeing zombie application package in the store, which taking most of the disk space.
- Verify whether meta data for the older version is associated with ISS [ Get-SFImageStoreContent -remoterelativepath ‘Store\IberFabric.App.PortalComercial.Services.AgendaType’ verify whether older package type is in the list ]
- If the older file is not listed in the #1, you can rdp into each VM which hosting the ISS replica, and delete it manually
- If meta data is present in #1 output, please verify the version customer wanted to delete is already in the registry by running, ‘Get-ServiceFabricApplicationType’
- In the #3 output, If the application type is already there, please un provision by ‘Unregister-ServiceFabricApplicationType’
- If the #3 output application type detail is not present, you can run ‘ Remove-ServiceFabricApplicationPackage -ApplicationPackagePathInImageStore "Store\IberFabric.App.PortalComercial.Services.AgendaType\<<package with version number>>
[https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-application-upgrade-tutorial-po...]
https://docs.microsoft.com/en-us/powershell/module/servicefabric/register-servicefabricapplicationty...
Note : For 6.5+ Cluster, SFX had ImageStoreService tab in Cluster, where GetSize button to get the size of Store content.
