This post was authored by Mani Ramaswamy, a Program Manager on the Azure Service Fabric team.
At //build 2016, we announced a limited preview of Service Fabric on Linux, enabling you to build, deploy, and manage cloud applications using the same core technology that Microsoft has used to build massively scalable solutions on Windows. This blog discusses how we got the product running on Linux, some of the forks we had to take and how we made those decisions – in other words, an inside view on the port. If you are looking for a guide/tutorial, please sign up for our preview at http://aka.ms/servicefabriclinuxpreview .
Linux is hardly a newcomer to Azure. In fact, one out of every four Azure virtual machines run Linux today and the Azure Marketplace offers more than a thousand community-provided Linux images, in addition to partner provided images. During its lifetime as a platform for running internal Microsoft services, however, Linux was naturally not a priority for Service Fabric. Once we decided to make the technology available to the public, though, it became a critical part of ensuring that it would be available everywhere, in an infrastructure and OS-agnostic way. In July 2015, we kicked off the effort, targeting Ubuntu (with other distros to follow).
A lot of our runtime is written in native C++, while some of the system services as well as our programming models have fair amounts of .NET managed code. Our system services have state (so things like failover work seamlessly), and we use ESE on Windows. On Linux, we started with Firebird , an OSS DB as our interim DB.
Once we had enough of the native code ported, so that we could get a cluster up, we started work on the programming models – reliable services and reliable actors. While the fastest path would have been to enable our existing C# APIs on .NET Core, we chose to meet Linux developers where they are by providing support for Java first. C# programming models on Linux are being worked on as we speak, and should be available shortly – since our product on Windows relies on the full CLR, there’s a fair amount of work to get our programming models to work on C# on CoreCLR which is what’s available on Linux.
While Stateless services support were quickly brought up, the Reliable Stateful Services that we offer on Windows use a store that we internally refer to as TStore – this had a significant amount of C# code. Hence, we started off a parallel effort to re-write TStore in C++ with light-weight managed wrappers that will be exposed as the programming models. This effort is proceeding as of this writing, and is expected to take a few months. However, Reliable Actors on Windows was built to work with both TStore as well as ESE on Windows. Since we already had written a layer on top of Firebird to bootstrap Service Fabric system services on Linux, we re-wrote Reliable Actors in Java, so our preview customers can get a taste for stateful services. When TStore is available on Linux, we will make the switch to use TStore instead of Firebird in a transparent manner.
Linux support should mean developers can build applications using the tools that they’re used to. Thus, our preview includes support for Yeoman scaffolding and Gradle builds and the SDK packages can be acquired via apt-get. We integrated with the Azure CLI on Linux, so you can use a bash shell to operate and manage your applications and cluster (we use PowerShell on Windows). And while we work on enabling a native development experience on Mac, we are providing a Vagrant-based setup so that you can easily stand up a local Service Fabric cluster in a VirtualBox VM.
When testing, we needed both a one-box cluster, as well as real multi-machine clusters. We integrated the latter with the portal, so our preview customers can create Linux SF clusters through the portal. The underlying infrastructure used is the same across Windows and Linux; for example, your cluster is built on the VM Scale Sets that can be automatically scaled up or down based on load.
Figure 3: Create Service Fabric Linux Clusters through the Azure portal
Of course, we couldn’t be a credible distributed application platform without supporting containers, so we enable running arbitrary “guest containers”, as well as Service Fabric stateless services within containers. We are tweaking this for a better tooling experience using Docker containers.
So this is where we are – Reliable Actors and Stateless Reliable Services in Java on top of our runtime, with an E2E install and SDK experience with support for containers and guest executables, and the ability to create and deploy to a cluster on Azure, or on your one-box.
Going forward, we are working on adding support for C# programming models along with adding features to bring the product close to parity with Windows. We haven’t yet decided whether we should wait until the TStore port is done to support Stateful Services on C# and Java, or if we should pursue an interim solution. Meanwhile, feel free to join the preview and give our bits a try. We’re looking not only for feedback to improve the product, but to influence the future direction on what it takes – participate in the preview to make your voice heard!