SOLVED
Home

enabling s2d cluster in virtual test environment issue

%3CLINGO-SUB%20id%3D%22lingo-sub-54255%22%20slang%3D%22en-US%22%3Eenabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-54255%22%20slang%3D%22en-US%22%3E%3CP%3EHello%2C%3C%2FP%3E%3CP%3EI%20am%20trying%20to%20setup%20a%20s2d%20cluster%20within%20a%20virtual%20hyper-v%20hyper-converged%20setup%20with%20two%20vms%2C%20but%20when%20attempting%20to%20enable-clusters2d%20it%20gets%20hung%20up%20at%2027%25%20with%20%22Waiting%20until%20SBL%20disks%20are%20surfaced%22%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAny%20ideas%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ethx%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-54255%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ESoftware%20Defined%20Data%20Center%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-346272%22%20slang%3D%22en-US%22%3ERe%3A%20enabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-346272%22%20slang%3D%22en-US%22%3ESorry%20this%20was%20a%20while%20ago%2C%20and%20I%20no%20longer%20am%20using%20the%20test%20environment.%20I%20now%20have%20two%20working%20production%20clusters.%20Late%20response%20unfortunately.%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-242967%22%20slang%3D%22en-US%22%3ERe%3A%20enabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-242967%22%20slang%3D%22en-US%22%3E%3CP%3EWhen%20you%20recreated%20the%20VHDX%20files%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E1.%20Did%20you%20create%20them%20all%20individually%3F%26nbsp%3B%20Or%2C%20did%20you%20create%20one%20and%20copy%20it%20to%20all%20the%20others%3F%26nbsp%3B%20Or%20some%20combination%3F%3C%2FP%3E%0A%3CP%3E2.%20Are%20these%20fixed%20or%20dynamically%20expanding%20VHDX%20disks%3F%3C%2FP%3E%0A%3CP%3E3.%20Are%20you%20using%20Windows%20Server%202016%20or%202019%3F%26nbsp%3B%20In%20Windows%20Server%202019%2C%20there%20is%20an%20additional%20.MHT%20file%20for%20enabling%20S2D%20that%20may%20give%20you%20some%20hints%20as%20to%20the%20issue.%3C%2FP%3E%0A%3CP%3E4.%20Did%20all%20the%20S2D%20tests%20in%20Validation%20pass%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-242885%22%20slang%3D%22en-US%22%3ERe%3A%20enabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-242885%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F170613%22%20target%3D%22_blank%22%3E%40Doug%20Swanson%3C%2FA%3E%20Did%20you%20get%20this%20resolved%3F%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHow%20are%20the%20VHDX%20files%20being%20exposed%20to%20the%20S2D%26nbsp%3BVM%20Nodes%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-237900%22%20slang%3D%22en-US%22%3ERe%3A%20enabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-237900%22%20slang%3D%22en-US%22%3E%3CP%3EI%20am%20having%20the%20same%20issue....%26nbsp%3B%20I%20have%20tried%20it%20several%20times....%20each%20time%20with%20new%20VM%20Builds%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CUL%3E%3CLI%3EMy%20Lab%20environment%20-%20Server%202019%20DC%20and%20HyperV%3C%2FLI%3E%3CLI%3EHP%20DL360%20G7%20288%20GB%20Ram%3C%2FLI%3E%3CLI%3EStorage%20Controllers%20P411e%20for%20the%204%202.5%22%2010K%20300GB%20HDD%2C%20P411%20for%20the%20second%20MSA60%2C%20SAS9200-8e%20for%20the%20first%20MSA60%3C%2FLI%3E%3CLI%3EStorage%20Arrays%2C%202%20MSA60%3C%2FLI%3E%3CLI%3EI%20am%20creating%202%20VMs%20and%26nbsp%3Battach%20physical%26nbsp%3Bdrives%20from%20the%20MSA60%20using%20the%20SAS9200-8e%20controller%26nbsp%3B%3C%2FLI%3E%3CLI%3EVM1%26nbsp%3B%202-480GB-SSD%2C%204%203TB-HDD%3C%2FLI%3E%3CLI%3EVM1%26nbsp%3B%202-480GB-SSD%2C%204%203TB-HDD%3C%2FLI%3E%3CLI%3EI%20used%20Get-PhysicalDisk%20to%20verify%20all%20of%20the%20disks%20are%20up%20and%20there%20are%20no%20partitions%3C%2FLI%3E%3CLI%3E%3CSPAN%20class%3D%22hljs-pscommand%20x-hidden-focus%22%3EOnce%20Verified%20I%20create%20the%20cluster%20%3C%2FSPAN%3E%3C%2FLI%3E%3CLI%3E%3CSPAN%20class%3D%22hljs-pscommand%20x-hidden-focus%22%3ENew-Cluster%3C%2FSPAN%3E%3CSPAN%20class%3D%22hljs-parameter%22%3E%20-Name%3C%2FSPAN%3E%26nbsp%3BMyS2DCluster%3CSPAN%20class%3D%22hljs-parameter%22%3E%20-Node%3C%2FSPAN%3E%20MyS2D1%2CMyS2D2%3CSPAN%20class%3D%22hljs-parameter%20x-hidden-focus%22%3E%20-StaticAddress%3C%2FSPAN%3E%20192.168.10.13%3C%2FLI%3E%3CLI%3EVerify%20the%20Cluster%3C%2FLI%3E%3CLI%3EThen%20run%20Enable-ClusterStorage%20Spaces%20Direct%20-Verbose%3C%2FLI%3E%3CLI%3EIt%20runs%20until%2027%25%20and%20hangs%20on%20Waiting%20until%20SBL%20disks%20are%20surfaced...%3C%2FLI%3E%3C%2FUL%3E%3CP%3EHere%20is%20the%20Verbose%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EPS%20C%3A%5CWindows%5Csystem32%26gt%3B%20Enable-ClusterStorageSpacesDirect%20-Verbose%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A13.619%20Ensuring%20that%20all%20nodes%20support%20S2D%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A13.650%20Querying%20storage%20information%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A15.578%20Sorted%20disk%20types%20present%20(fast%20to%20slow)%3A%20SSD%2CHDD.%20Number%20of%20types%20present%3A%202%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A15.578%20Checking%20that%20nodes%20support%20the%20desired%20cache%20state%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A15.578%20Checking%20that%20all%20disks%20support%20the%20desired%20cache%20state%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A26.961%20Creating%20health%20resource%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A32.125%20Setting%20cluster%20property%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A32.125%20Setting%20default%20fault%20domain%20awareness%20on%20clustered%20storage%20subsystem%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A32.344%20Waiting%20until%20physical%20disks%20are%20claimed%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A35.360%20Number%20of%20claimed%20disks%20on%20node%20'MyS2D1'%3A%206%2F6%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A38.422%20Number%20of%20claimed%20disks%20on%20node%20'MyS2D2'%3A%206%2F6%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A38.438%20Node%20'MyS2D1'%3A%20Waiting%20until%20cache%20reaches%20desired%20state%20(HDD%3A'ReadWrite'%20SSD%3A'WriteO%3C%2FP%3E%3CP%3Enly')%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A38.453%20SBL%20disks%20initialized%20in%20cache%20on%20node%20'MyS2D1'%3A%206%20(6%20on%20all%20nodes)%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A38.453%20Cache%20reached%20desired%20state%20on%20MyS2D1%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A39.469%20Node%20'MyS2D2'%3A%20Waiting%20until%20cache%20reaches%20desired%20state%20(HDD%3A'ReadWrite'%20SSD%3A'WriteO%3C%2FP%3E%3CP%3Enly')%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A39.485%20SBL%20disks%20initialized%20in%20cache%20on%20node%20'MyS2D2'%3A%206%20(12%20on%20all%20nodes)%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A39.485%20Cache%20reached%20desired%20state%20on%20MyS2D2%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A40.505%20Waiting%20until%20SBL%20disks%20are%20surfaced%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A43.552%20Disks%20surfaced%20on%20node%20'MyS2D1'%3A%209%2F12%3C%2FP%3E%3CP%3EVERBOSE%3A%202018%2F08%2F26-16%3A25%3A43.599%20Disks%20surfaced%20on%20node%20'MyS2D2'%3A%209%2F12%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-84186%22%20slang%3D%22en-US%22%3ERe%3A%20enabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-84186%22%20slang%3D%22en-US%22%3E%3CP%3ESee%20this%20blog%20which%20outlines%20the%20process%20for%20creating%20a%20guest%20cluster%20with%20S2D%3A%3CBR%20%2F%3E%3CA%20href%3D%22https%3A%2F%2Fblogs.msdn.microsoft.com%2Fclustering%2F2015%2F05%2F27%2Ftesting-storage-spaces-direct-using-windows-server-2016-virtual-machines%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fblogs.msdn.microsoft.com%2Fclustering%2F2015%2F05%2F27%2Ftesting-storage-spaces-direct-using-windows-server-2016-virtual-machines%2F%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWhat%20hypervisor%20are%20you%20running%20the%20VMs%20on%3F%26nbsp%3B%26nbsp%3B%20Hyper-V%3F%26nbsp%3B%20VMware%3F%26nbsp%3B%20Azure%3F%26nbsp%3B%20AWS%3F%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThanks!%3CBR%20%2F%3EElden%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-83814%22%20slang%3D%22en-US%22%3ERe%3A%20enabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-83814%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EMy%20solution%20to%20this%20when%20I%20worked%20on%20it%20was%20to%20disable-cluster2d%2C%20then%20I%20re-created%20the%20vhdx%20files%2C%20and%20enable-cluster2d%20again.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ethx%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-80991%22%20slang%3D%22en-US%22%3ERe%3A%20enabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-80991%22%20slang%3D%22en-US%22%3EAre%20you%20trying%20to%20set%20this%20up%20as%20a%20lab%20between%20two%20VMs%3F%20If%20so%20I%20have%20found%20this%20can%20take%20up%20to%2015%20plus%20minutes%20depending%20on%20your%20lab%20hardware.%20Just%20ran%20into%20this%20again%20today%20be%20patient%20it%20isn't%20hung.%20Make%20sure%20you%20add%20the%20-verbose%20switch%20to%20enable-clusters2d%3CBR%20%2F%3E%3CBR%20%2F%3ESo%20go%20grab%20a%20coffee%20and%20try%20again.%20Also%20if%20it%20half%20configured%20there%20is%20a%20script%20on%20the%20technet%20gallery%20called%20clear-sdsconfig.ps1%20they%20wipes%20the%20cluster%20config%20and%20disks%20and%20lets%20you%20start%20over.%3CBR%20%2F%3E%3CBR%20%2F%3EThanks%2C%3CBR%20%2F%3E%3CBR%20%2F%3EDave%20kawula%20MVP%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-80988%22%20slang%3D%22en-US%22%3ERe%3A%20enabling%20s2d%20cluster%20in%20virtual%20test%20environment%20issue%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-80988%22%20slang%3D%22en-US%22%3E%3CP%3EDell%3F%20Firmware.%20Make%20sure%20your%20firmware%20is%20up%20to%20date.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWhat%20is%20the%20East-West%20fabric%20(between%20nodes)%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
Robert Lien
Contributor

Hello,

I am trying to setup a s2d cluster within a virtual hyper-v hyper-converged setup with two vms, but when attempting to enable-clusters2d it gets hung up at 27% with "Waiting until SBL disks are surfaced"

 

Any ideas?

 

thx

8 Replies

Dell? Firmware. Make sure your firmware is up to date.

 

What is the East-West fabric (between nodes)?

Are you trying to set this up as a lab between two VMs? If so I have found this can take up to 15 plus minutes depending on your lab hardware. Just ran into this again today be patient it isn't hung. Make sure you add the -verbose switch to enable-clusters2d

So go grab a coffee and try again. Also if it half configured there is a script on the technet gallery called clear-sdsconfig.ps1 they wipes the cluster config and disks and lets you start over.

Thanks,

Dave kawula MVP

Hi,

 

My solution to this when I worked on it was to disable-cluster2d, then I re-created the vhdx files, and enable-cluster2d again.

 

thx

See this blog which outlines the process for creating a guest cluster with S2D:
https://blogs.msdn.microsoft.com/clustering/2015/05/27/testing-storage-spaces-direct-using-windows-s...

 

What hypervisor are you running the VMs on?   Hyper-V?  VMware?  Azure?  AWS?

 

Thanks!
Elden

I am having the same issue....  I have tried it several times.... each time with new VM Builds

 

  • My Lab environment - Server 2019 DC and HyperV
  • HP DL360 G7 288 GB Ram
  • Storage Controllers P411e for the 4 2.5" 10K 300GB HDD, P411 for the second MSA60, SAS9200-8e for the first MSA60
  • Storage Arrays, 2 MSA60
  • I am creating 2 VMs and attach physical drives from the MSA60 using the SAS9200-8e controller 
  • VM1  2-480GB-SSD, 4 3TB-HDD
  • VM1  2-480GB-SSD, 4 3TB-HDD
  • I used Get-PhysicalDisk to verify all of the disks are up and there are no partitions
  • Once Verified I create the cluster
  • New-Cluster -Name MyS2DCluster -Node MyS2D1,MyS2D2 -StaticAddress 192.168.10.13
  • Verify the Cluster
  • Then run Enable-ClusterStorage Spaces Direct -Verbose
  • It runs until 27% and hangs on Waiting until SBL disks are surfaced...

Here is the Verbose 

 

PS C:\Windows\system32> Enable-ClusterStorageSpacesDirect -Verbose

VERBOSE: 2018/08/26-16:25:13.619 Ensuring that all nodes support S2D

VERBOSE: 2018/08/26-16:25:13.650 Querying storage information

VERBOSE: 2018/08/26-16:25:15.578 Sorted disk types present (fast to slow): SSD,HDD. Number of types present: 2

VERBOSE: 2018/08/26-16:25:15.578 Checking that nodes support the desired cache state

VERBOSE: 2018/08/26-16:25:15.578 Checking that all disks support the desired cache state

VERBOSE: 2018/08/26-16:25:26.961 Creating health resource

VERBOSE: 2018/08/26-16:25:32.125 Setting cluster property

VERBOSE: 2018/08/26-16:25:32.125 Setting default fault domain awareness on clustered storage subsystem

VERBOSE: 2018/08/26-16:25:32.344 Waiting until physical disks are claimed

VERBOSE: 2018/08/26-16:25:35.360 Number of claimed disks on node 'MyS2D1': 6/6

VERBOSE: 2018/08/26-16:25:38.422 Number of claimed disks on node 'MyS2D2': 6/6

VERBOSE: 2018/08/26-16:25:38.438 Node 'MyS2D1': Waiting until cache reaches desired state (HDD:'ReadWrite' SSD:'WriteO

nly')

VERBOSE: 2018/08/26-16:25:38.453 SBL disks initialized in cache on node 'MyS2D1': 6 (6 on all nodes)

VERBOSE: 2018/08/26-16:25:38.453 Cache reached desired state on MyS2D1

VERBOSE: 2018/08/26-16:25:39.469 Node 'MyS2D2': Waiting until cache reaches desired state (HDD:'ReadWrite' SSD:'WriteO

nly')

VERBOSE: 2018/08/26-16:25:39.485 SBL disks initialized in cache on node 'MyS2D2': 6 (12 on all nodes)

VERBOSE: 2018/08/26-16:25:39.485 Cache reached desired state on MyS2D2

VERBOSE: 2018/08/26-16:25:40.505 Waiting until SBL disks are surfaced

VERBOSE: 2018/08/26-16:25:43.552 Disks surfaced on node 'MyS2D1': 9/12

VERBOSE: 2018/08/26-16:25:43.599 Disks surfaced on node 'MyS2D2': 9/12

 

 

@Doug Swanson Did you get this resolved?

 

How are the VHDX files being exposed to the S2D VM Nodes?

When you recreated the VHDX files,

 

1. Did you create them all individually?  Or, did you create one and copy it to all the others?  Or some combination?

2. Are these fixed or dynamically expanding VHDX disks?

3. Are you using Windows Server 2016 or 2019?  In Windows Server 2019, there is an additional .MHT file for enabling S2D that may give you some hints as to the issue.

4. Did all the S2D tests in Validation pass?

Solution
Sorry this was a while ago, and I no longer am using the test environment. I now have two working production clusters. Late response unfortunately.
Related Conversations