HCI - Storage Spaces Direct - Two nodes

Copper Contributor

Find a powerful, economical HCI solution that is suitable for small and medium environments.


The idea :

1. a small hyperconverged server with 4 hot-swap 2.5 NVMe + bi proc + 4x10Gbps (rdma support) + 2 M.2 ssd for os + etc... / per Node

1.GIF

 

----------------------------

 

2. A good adapted sizing for Storage Space Direct

 

plan_stocl.png

 

---------------------------------

 

3. Configure Quorum of this little cluster on Azure.

 

 ---

Your opinion on the solution ? :)

 

 

 

 

 

 

 

 

 

 

EDIT --> 18/06/2020 : HCI - Cluster - Storage Spaces Direct - Two nodes (V2)

4 Replies

Hyper-Converged Infrastructure (HCI) is all the rage these days, but this doesn't make it a universal solution. If you can afford it, great. My only problem with two nodes is that you lose half your storage capacity. Well, not lose, because you get redundancy, but at 4 nodes or higher you start to get better storage efficiency.

Here's a nice and short story about something similar to your configuration: Project Kepler.

If you need more links on the topic (with tutorials etc), ping me.

Cheers. Emanuel

I tried to consider a 4-node infrastructure. But this solution exceeds my budgets.

Processors, memory, network cards, WinServ licenses, everything must be doubled against a 2-node solution.

For once, the loss of storage capacity remains more advantageous.

With this solution, I think I can demonstrate that in production, the use of two nodes is not necessarily more risky if they are prepared and configured correctly.

 

But effectively, the sizing choices must be in correlation with the production environment.

 

In my case, the environment looks like this:

1st node: SRV-DC, SRV-FILE, SRV-DA, SRV-BD, SRV-APP-v, SRV-APP, etc.

2nd node: SRV-ADM, SRV-WSUS, SRV-PRTG, SRV-TEST, SRV-VEEAM, etc.

If one of the nodes falls, I'm sure to keep excellent performance.

 

NVMe technology, coupled with fast network interfaces, must assure production performance but must also be the mainstays of my backup system !

It's also a perfect technology for Veeam's WAN Accelerator cache.

My Backup system... which is also another interesting project since I'm giving up the notion of archiving and dedicated myself on long-term backup !

Yes Yes, I just want to be sure that in 10 years my saved data will be usable quickly. This is only rarely the case with classical archiving.

 

For the Kepler-47 project, the idea of using the Thunderbolt techno intrigues me (real performance, stability) I will not be against a little more info on the subject :smiling_face_with_smiling_eyes:.

QCD report can be excellent.

I really want more info !

 

Thank’s for your answer Emanuel.

And I'm waiting for your interesting documentation.

 

Jean-Charles

Well, quite a lengthy answer, I’ll try to answer some of the points you brought up.

Beancounters are an important factor to take into consideration, too. As long as you can afford to get the redundant storage, you should be fine. However, unless a single node can sustain the entire load of all the VMs you won’t be able to do things like patching during production hours. Reading your message a second time, I noticed you covered this. :)

I was thinking on installing at least one local (not clustered) DC as a VM on one of the nodes (possibly two, as I have four in total). Unless you have another DC beyond those two, it will be a lengthy and major pain to restore in case you have an AD outage. Try to keep at least a DC VM as a single DC, and nothing else (and DNS, of course). If you have so many roles, it’ll take a lot of time to restore and clean-up everything.

I am not a storage expert, but backup on disk is definitely faster and less painful. I don’t know about agents for specific apps (which may provide faster and more granular restore options for a database or CRM or whatever), but at least for Windows infrastructure Windows Server backup is good and reliable. Since it no longer works on tape, a bunch of disks would be great (I plan to use the old storage for that).  However, if you plan to ditch the tapes, make sure you’re using a more sophisticated file-system (that can handle bit rot, for instance). If you still have some tape library laying around, you could still do a less often backup for things that really matter.

Moving on to Kepler. It’s not my project, so I cannot not give you more number than what I’ve read in the post.

 

As I’ve struggled a lot with S2D (I’m not a storage guy, and not a great networking guy, either), this step-by-step article for 2-nodes HCI helped me a lot to clarify some fuzzy concepts.

https://www.tech-coffee.net/2-node-hyperconverged-cluster-with-windows-server-2016/

And now for some more performance-related links. As some of them are from vendors, they may be useful but take them with the proverbial grain of salt:

https://docs.microsoft.com/en-us/windows-server/storage/storage-spaces/performance-history

https://docs.microsoft.com/en-us/windows-server/administration/performance-tuning/subsystem/storage-...

https://www.micron.com/about/blogs/2017/september/microsoft-storage-spaces-direct-is-an-io-performan...

Also, https://youtu.be/raeUiNtMk0E?t=274 for some numbers.

There was another post from a Microsoft PFE, in which I remember the last picture of the post was a screenshot showing 1GBps copy between nodes, but I couldn’t find the link.

 

I hope you found at least some of these bits helpful.

Cheers. Emanuel

 

Hi people,

 

After hours and hours of research, I think I found a HCI solution that fits everyone, or at least for SME.

I've steer my choice on brand SuperMicro because their solutions are more flexible.

I suggest you to pay attention to the cost of the NVME SSD.
There is nothing clear about explanation of their sharp price hikes !
For example, for this reference "Intel DC P4500 2 To NVMe (SSDPE2KX020T701)". About three weeks ago the price for this ref was 30% cheaper !

 

INFRASTRUCTURE HYPER-CONVERGÉE
DésignationCaractéristiquesref.Quantité
Serveur SuperMicro- Dual Socket P (LGA 3647) | Intel® Xeon® Scalable Processors
- Memory 2666/2400/2133MHz ECC DDR4 SDRAM | 2666 ECC DDR4 NVDIMM
- Intel® C621 chipset
- 2 x Sata Dom / SuperDOM
- 2 x 10GBase-T ports via AOC-URN6-i2XT |Intel® X540 Dual Port 10GBase-T
- IPMI 2.0 with virtual media over LAN and KVM-over-LAN support
- Graphic ASPEED AST2500 BMC
- 10 Hot-swap 2.5" Drive Bays; 10 NVMe (4 hybrid ports)
- 8 x Heavy duty fans
- 2 x 1000W Redundant Power Supplies | Titanium Level (96%)
- 2 x  Passive CPU Heat Sink
- 2 x Rack mount rails
SYS-1029U-TN10RT2
OPTIONS DE CONFIGURATION
Puce TPM SuperMicroSuperMicro Crypto-processeur / TPM 2.0 with Infineon 9670 controllerAOM-TPM-9670H2
ProcesseurIntel® Xeon® Silver 4116  | 16.5 MB L3, 12 Cœurs, 24 Fils | LGA 3647 | 2.1GHzBX8067341164
MémoireCrucial DDR4 | 8 Go | Reg ECC | Rank : 1 | Proc : 2666 MHz | Minuterie : CL19CT8G4RFS826632
StockageIntel SSD 2.5" DC P4500 2 To NVMe lec:3,3Go/s écr:1,2Go/s   515 000 IOPS   Gar. 5 ansSSDPE2KX020T70110
internal M.2 NVMe cardSupermicro PCIe Add-On Card for up to two NVMe SSDsAOC-SLG3-2M22
SSD M.2SSD 256 Go Intel Pro 7600p M.2 PCIe NVMe SSDPEKKF256G8X14
Carte EthernetMellanox ConnectX-4 Lx 2x 25Gbps RDMA over Converged Ethernet (RoCE) SFP28AOC-S25G-m2S2
Cable SFPSuperMicro 1.5m 25GbE SFP28 to SFP28, PassiveCBL-NTWK-0944-MS28C15M2
SuperDom SSDSuperMicro 64 Go, mSATA, Série ATA III, 520 Mo/s, 6 Gbit/s) SSD-DM064-SMCMVN14

 

If you need clarification on this conf, let me know!