Published
Aug 06 2019 05:21 PM
1,537
Views
First published on MSDN on Dec 10, 2018
This article was written by Jason Haley , an independent consultant and Microsoft Azure MVP.
In the past couple of years, application developers have stopped questioning whether microservices are the right architecture to build scalable systems and started debating how to best implement them. I think this is due to several concepts that have converged in our industry in the past decade—all of which have to do with the need for businesses to bring innovation to their customers faster and faster.
It has now been a couple of decades since the rise of agile caused the slow decline of waterfall and its motivations have permeated all aspects of creating business applications. Agile began moving the business and development organizations closer, improved communication, started automated testing, and quickly delivered working software as a measure of progress. Then came the combining of development and operations, better known as DevOps, that brought continuous integration (CI), continuous delivery (CD), and containers as a delivery format. These all help shorten the feedback loop between the business and the delivered software.
The next natural step was to evolve application architecture to microservices, where the application is divided into small services that are modeled around the business domain and can be independently deployed. Soon after adopting microservices, we are faced with a new challenge: we need a platform to manage all these services.
Today, we will take a high-level look at microservices platform needs, get an introduction to Azure Service Fabric Mesh, and then set up a CI/CD pipeline to deploy to Azure Service Fabric Mesh.
The whole idea of using a microservices platform is being able to spend your time building a great microservices application and not stitching together disparate systems that will be difficult to use and operate.
A microservices platform should:
Also depending on your application’s implementation, you may need to keep in mind OS platform support, e.g. if you need Windows containers, does the platform support them?.
When you start looking at microservices platforms, you’ll need to determine what type of hosting is good for you.
Some questions to consider:
When answering these questions, you should also keep in mind the maintenance and operational cost involved when it comes to hosting a cluster yourself.
If you do need full control (example: you must run a custom server image or virus scanning software), knowing that ahead of time will help you narrow your options. Some microservices platforms, like Azure Service Fabric , can be run on-premises or in the cloud.
However, if you don’t need full control and don’t want to hire an operations staff to babysit the cluster, Azure Service Fabric Mesh is a fully managed platform.
Earlier this year at the Build 2018 conference, the Azure Service Fabric team introduced their new Azure Service Fabric Mesh offering. Service Fabric Mesh joins the Service Fabric product line as the fully managed microservices platform. The service has been in public preview since July 2018 and is available in four Azure regions: East Asia, US West, US East, and Europe West. General availability is currently planned for 2019.
Image from Ignite 2018, “ Azure Service Fabric overview and the road ahead .” Courtesy of Mark Fussell @mfussell
Service Fabric Mesh is a fully managed microservices platform built on top of Service Fabric clusters. The complexity of the underlying cluster is hidden from you. You just need to describe your application, its services, network, storage, etc., and the platform will make it happen. The one requirement is that your application’s services do need to run in containers. Both Linux and Windows containers are supported.
Service Fabric Mesh is designed to simplify the experience of a microservices platform. The key capabilities are exposed by the Service Fabric resource model .
Service Fabric Mesh uses a resource model to define the resources that make up an application. Each resource is declared in a resource file that is used to deploy and manage the application. Both YAML and JSON formats are supported. The resource model is a first-class citizen in Azure Resource Manager (ARM).
In the resource model, the following types are supported or soon will be:
1 On the roadmap at Ignite 2018.
2 Available in the Dec 2019 Service Fabric Mesh update.
More details and documentation are available in the Azure Service Fabric Mesh documentation.
Now it’s time to get hands-on and deploy an application to Service Fabric Mesh. The Service Fabric Mesh team has several samples to get started with on GitHub at service-fabric-mesh .
In this tutorial, you’ll work with the VotingApp sample. The voting application has two services--one for a web UI and the other a back-end service. The goals are to:
You need to first clone or download the samples repository . The easiest way is to go to your command line and type the following command:
Once you have the code locally, open the VotingApp.sln solution file.
In the Solution Explorer, you’ll notice the VotingApp project. That is the Service Fabric Mesh project, which contains the YAML files for the application level resources.
There are also service.yaml files in the individual web projects that contain the service-level configuration.
The YAML file is what describes the resource model mentioned earlier. It is the description of resources in your application that Service Fabric Mesh will use to create and manage the infrastructure.
Before we push our code to new repository, we need to make a few small changes to the Dockerfiles. The changes are due to the build server not currently being compatible with the base image used in the original Dockerfiles and some paths I changed for the build.
In the VotingData project, find the Dockerfile and modify it to be the following:
In the VotingWeb project, find the Dockerfile and modify it to be the following:
Save and commit your changes.
Before we can create a CI/CD pipeline, we need to take a small detour and get the code into an Azure DevOps repository.
In the Team Explorer , click the Settings button and then the Repository Settings .
At the bottom of the panel, you will see the Remotes are mapped to the original GitHub repo. Click the Remove link and then OK for the confirmation dialog.
Now that it is disconnected from GitHub, go back to the Team Explorer Home and click the Sync button.
The Push view will now have a Publish Git Repo button. Click it.
Once Visual Studio has determined your Azure DevOps account, you will have the following view where you need to select the account, give a repository name you want to use, and click Publish Repository .
The whole repository you cloned from GitHub will be published to the new repository. Click the link for your repository.
Next, we can look at the ARM template.
We want to build our own containers from the source and create a release pipeline to deploy them to Service Fabric Mesh. To do that, we need to modify the ARM template for the voterapp.
You should now have your Azure DevOps repository open in the browser. In the menu on the left, choose Repo -> Files and locate the voting mesh_rp.windows.json under voterapp/templates/voting and open it.
The ARM template is made up of parameters and resources. When Azure takes the ARM template, it will use the parameters to fill in the template before creating/updating resources.
We are going to be building Docker images and putting them in ACR. To do this, we need to add the following parameters:
Click the Edit button at the top of the view to start editing your template. Modify your parameters section to contain the following:
Now, look through the resources and find the VotingData service and its codePackages . You’ll need to modify the image and add the imageRegistryCredential shown below:
Next, you need to make similar changes to the VotingWeb service’s codePackages . Modify the image and add the imageRegistryCredential shown below:
Now click the Commit button at the top of the editor and then click commit on the dialog box to save the changes.
The template is now parameterized and ready to use in a build pipeline.
The build has the responsibility to take the source and create the artifacts needed for deployment. To create a new build pipeline, select the Pipelines -> Builds and then the New pipeline button.
On the New pipeline view, click the Use the visual designer link.
Verify that the information is correct on the Select a source view and click Continue .
On the Select a template view, select Docker container and click Apply.
On the Pipeline view, make sure that Hosted VS2017 is selected for the agent.
Clone both the Build an image task and Push an Image task by right-clicking and select Clone task(s) .
Then use the Add a task button to add Copy Files to and Publish Artifact tasks to the pipeline.
Click the first Build an image task and change the settings to match the following (using your Azure Subscription and Azure Container Registry):
On the first Push an image task, change the settings to push the votingweb image to your ACR.
Repeat the same steps for the second Build an image and Push an image tasks.
The Build an image task for the votingdata image should look as follows:
Push an image for the votingdata image should look as follows:
Modify the Copy Files task to look like the following:
Verify the Publish Build Artifacts are configured as follows:
Select the Triggers tab and check Enable continuous integration .
Click the Save button in the top left corner and then again on the dialog to save the build.
The responsibility of the release is to take the artifacts the build created and deploy to an environment.
In the left menu, select Releases and the New pipeline button.
On the Select a template view, click the
Empty job
link.
Now click the Add an artifact button, to configure where the artifacts are coming from.
Modify the settings to match the following and click Add :
Once the artifact is added, click the Continuous deployment trigger (lightning bolt) and enable the trigger.
This enables the release to start when a build completes.
Next, we need to add the variables to pass into the template. Remember all those parameters we added?
Click the Variables tab and use the Add button to create and populate the variables with your values.
When you are done, it should look like following:
The registry password, username, and server are for your ACR and can be retrieved from the Access keys area of your Azure Container Registry in the Azure portal.
We are now ready to create the release task.
Go to the Tasks tab and search for resource . Find the Azure Resource Group Deployment and click Add .
Modify the settings to reflect the following:
Click
Save
in the upper right corner and
OK
on the dialog to save the release pipeline.
Now all we need to do is make a change to the repo and the CI/CD process will start.
In the left menu, select Repos and Files , then click the README.md and the Edit button.
Make some small change and click the Commit button and Commit on the dialog.
Now go to the Pipelines and Builds to see your build status.
Once the build has succeeded, you will need to get the IP address of your service. Currently, the easiest way to do that is in the cloud shell.
Find the Public IP address for your application and open it in a browser. You should see a page like the following:
In this article, we’ve looked at the challenges a microservices architecture has, introduced the new Azure Service Fabric Mesh microservices platform, and explored how to build and deploy an application to Service Fabric Mesh. Now that you have a beginning CI/CD pipeline set up for a Service Fabric Mesh application, you can start to explore how easy it is to host your microservices with it. You can find more information on the documentation site .
To contact Jason find him on Twitter @haleyjason
This article was written by Jason Haley , an independent consultant and Microsoft Azure MVP.
In the past couple of years, application developers have stopped questioning whether microservices are the right architecture to build scalable systems and started debating how to best implement them. I think this is due to several concepts that have converged in our industry in the past decade—all of which have to do with the need for businesses to bring innovation to their customers faster and faster.
It has now been a couple of decades since the rise of agile caused the slow decline of waterfall and its motivations have permeated all aspects of creating business applications. Agile began moving the business and development organizations closer, improved communication, started automated testing, and quickly delivered working software as a measure of progress. Then came the combining of development and operations, better known as DevOps, that brought continuous integration (CI), continuous delivery (CD), and containers as a delivery format. These all help shorten the feedback loop between the business and the delivered software.
The next natural step was to evolve application architecture to microservices, where the application is divided into small services that are modeled around the business domain and can be independently deployed. Soon after adopting microservices, we are faced with a new challenge: we need a platform to manage all these services.
Today, we will take a high-level look at microservices platform needs, get an introduction to Azure Service Fabric Mesh, and then set up a CI/CD pipeline to deploy to Azure Service Fabric Mesh.
What should a microservices platform provide?
The whole idea of using a microservices platform is being able to spend your time building a great microservices application and not stitching together disparate systems that will be difficult to use and operate.
A microservices platform should:
- Treat microservices as a first-class citizen
- Be able to scale services independently
- Have options for network communications (ingress and between services)
- Be secure and compliant
- Provide monitoring and auditing
- Be able to upgrade services with no downtime
- Be capable of using CI/CD pipelines
- Have options to store state (some services will need to keep state)
- Not need a team of dedicated engineers to maintain the system
Also depending on your application’s implementation, you may need to keep in mind OS platform support, e.g. if you need Windows containers, does the platform support them?.
The infrastructure spectrum: Full control to fully managed
When you start looking at microservices platforms, you’ll need to determine what type of hosting is good for you.
Some questions to consider:
- Do you need to run it on-premises?
- Do you need full control of the cluster?
- Do you have the infrastructure team to configure and run clusters of servers?
- Do you want a fully managed platform where you don’t need to maintain the underlying cluster?
- Does the platform need to be portable (able to run in multiple clouds or on-premises)?
When answering these questions, you should also keep in mind the maintenance and operational cost involved when it comes to hosting a cluster yourself.
If you do need full control (example: you must run a custom server image or virus scanning software), knowing that ahead of time will help you narrow your options. Some microservices platforms, like Azure Service Fabric , can be run on-premises or in the cloud.
However, if you don’t need full control and don’t want to hire an operations staff to babysit the cluster, Azure Service Fabric Mesh is a fully managed platform.
Azure Service Fabric Mesh
Earlier this year at the Build 2018 conference, the Azure Service Fabric team introduced their new Azure Service Fabric Mesh offering. Service Fabric Mesh joins the Service Fabric product line as the fully managed microservices platform. The service has been in public preview since July 2018 and is available in four Azure regions: East Asia, US West, US East, and Europe West. General availability is currently planned for 2019.
Image from Ignite 2018, “ Azure Service Fabric overview and the road ahead .” Courtesy of Mark Fussell @mfussell
Service Fabric Mesh is a fully managed microservices platform built on top of Service Fabric clusters. The complexity of the underlying cluster is hidden from you. You just need to describe your application, its services, network, storage, etc., and the platform will make it happen. The one requirement is that your application’s services do need to run in containers. Both Linux and Windows containers are supported.
Service Fabric Mesh is designed to simplify the experience of a microservices platform. The key capabilities are exposed by the Service Fabric resource model .
Resource model
Service Fabric Mesh uses a resource model to define the resources that make up an application. Each resource is declared in a resource file that is used to deploy and manage the application. Both YAML and JSON formats are supported. The resource model is a first-class citizen in Azure Resource Manager (ARM).
In the resource model, the following types are supported or soon will be:
- Applications are the units of deployment and versioning. They contain one or more services.
- Services are general properties about the host the containers need to run on and contain one or more code packages.
- Code package describes everything needed to run a container image, such as environment variables, CPU/memory, storage volumes, and endpoints.
- Networks contain the VNet properties the application is to be deployed in, as well as any ingress configuration. By default, a new VNet will be created for the application. The ability to use one of your existing VNets is coming soon. 1
- Volumes contain the general-purpose file storage to be used by the services: either Azure Files storage or Service Fabric Reliable Volume disk. 2
- Secrets describe values that will be either inline 2 or stored in Azure Key Vault 1 and accessed by a Managed Service Identity with Azure Active Directory (Azure AD) 1 .
- Gateways 2 are used to connect networks.
- Routing rules 1 are used to provide intelligent traffic routing.
- Auto-scale policy 2 sets thresholds for horizontal scaling to be triggered by CPU or memory limits.
1 On the roadmap at Ignite 2018.
2 Available in the Dec 2019 Service Fabric Mesh update.
More details and documentation are available in the Azure Service Fabric Mesh documentation.
Set up a CI/CD pipeline to deploy to Azure Service Fabric Mesh
Now it’s time to get hands-on and deploy an application to Service Fabric Mesh. The Service Fabric Mesh team has several samples to get started with on GitHub at service-fabric-mesh .
In this tutorial, you’ll work with the VotingApp sample. The voting application has two services--one for a web UI and the other a back-end service. The goals are to:
- Create a build that will start when a commit happens.
- Have the build create all the Docker images and put them in Azure Container Registry (ACR).
- Create a release that will start when the build completes.
- Have the release deploy the application to Service Fabric Mesh.
Prerequisites
- An Azure subscription; you can create a free account if you don’t have one.
- An Azure DevOps account; you can also create a free DevOps account if you don’t have one.
- Set up your development environment for creating Service Fabric Mesh apps.
- An Azure Container Registry; see this Quickstart on how to do that.
Get the code
You need to first clone or download the samples repository . The easiest way is to go to your command line and type the following command:
| git clone https://github.com/Azure-Samples/service-fabric-mesh.git |
Once you have the code locally, open the VotingApp.sln solution file.
In the Solution Explorer, you’ll notice the VotingApp project. That is the Service Fabric Mesh project, which contains the YAML files for the application level resources.
There are also service.yaml files in the individual web projects that contain the service-level configuration.
The YAML file is what describes the resource model mentioned earlier. It is the description of resources in your application that Service Fabric Mesh will use to create and manage the infrastructure.
Small housekeeping task
Before we push our code to new repository, we need to make a few small changes to the Dockerfiles. The changes are due to the build server not currently being compatible with the base image used in the original Dockerfiles and some paths I changed for the build.
In the VotingData project, find the Dockerfile and modify it to be the following:
|
FROM microsoft/aspnetcore:2.0-nanoserver-sac2016 AS base
WORKDIR /app FROM microsoft/aspnetcore-build:2.0 AS build WORKDIR /src COPY ./VotingData.csproj ./ RUN dotnet restore ./VotingData.csproj COPY . . WORKDIR /src RUN dotnet build VotingData.csproj -c Release -o /app FROM build AS publish RUN dotnet publish VotingData.csproj -c Release -o /app FROM base AS final WORKDIR /app COPY --from=publish /app . ENTRYPOINT ["dotnet", "VotingData.dll"] |
In the VotingWeb project, find the Dockerfile and modify it to be the following:
|
FROM microsoft/aspnetcore:2.0-nanoserver-sac2016 AS base
WORKDIR /app FROM microsoft/aspnetcore-build:2.0 AS build WORKDIR /src COPY ./VotingWeb.csproj ./ RUN dotnet restore ./VotingWeb.csproj COPY . . WORKDIR /src RUN dotnet build VotingWeb.csproj -c Release -o /app FROM build AS publish RUN dotnet publish VotingWeb.csproj -c Release -o /app FROM base AS final WORKDIR /app COPY --from=publish /app . ENTRYPOINT ["dotnet", "VotingWeb.dll"] |
Save and commit your changes.
Move the code in your own repository
Before we can create a CI/CD pipeline, we need to take a small detour and get the code into an Azure DevOps repository.
In the Team Explorer , click the Settings button and then the Repository Settings .
At the bottom of the panel, you will see the Remotes are mapped to the original GitHub repo. Click the Remove link and then OK for the confirmation dialog.
Now that it is disconnected from GitHub, go back to the Team Explorer Home and click the Sync button.
The Push view will now have a Publish Git Repo button. Click it.
Once Visual Studio has determined your Azure DevOps account, you will have the following view where you need to select the account, give a repository name you want to use, and click Publish Repository .
The whole repository you cloned from GitHub will be published to the new repository. Click the link for your repository.
Next, we can look at the ARM template.
Modify the ARM template
We want to build our own containers from the source and create a release pipeline to deploy them to Service Fabric Mesh. To do that, we need to modify the ARM template for the voterapp.
You should now have your Azure DevOps repository open in the browser. In the menu on the left, choose Repo -> Files and locate the voting mesh_rp.windows.json under voterapp/templates/voting and open it.
The ARM template is made up of parameters and resources. When Azure takes the ARM template, it will use the parameters to fill in the template before creating/updating resources.
Add parameters
We are going to be building Docker images and putting them in ACR. To do this, we need to add the following parameters:
- location – the Azure region for the Service Fabric Mesh cluster
- registry-server – the name of the ACR server
- registry-username – the username to access the ACR
- registry-password – the password to access the ACR
- web-image – the name of the web front-end image
- data-image – the name of the data service image
Click the Edit button at the top of the view to start editing your template. Modify your parameters section to contain the following:
|
"parameters": {
"location": { "type": "string" }, "registry-server": { "type": "string" }, "registry-username": { "type": "string" }, "registry-password": { "type": "securestring" }, "web-image": { "type": "string" }, "data-image": { "type": "string" } }, |
Now, look through the resources and find the VotingData service and its codePackages . You’ll need to modify the image and add the imageRegistryCredential shown below:
|
"name": "VotingData",
"properties": { "description": "VotingData backend service", "osType": "Windows", "codePackages": [ { "name": "VotingData", "image": " [parameters('data-image')] ", "imageRegistryCredential" : { "server" : "[parameters('registry-server')]" , "username" : "[parameters('registry-username')]" , "password" : "[parameters('registry-password')]" }, |
Next, you need to make similar changes to the VotingWeb service’s codePackages . Modify the image and add the imageRegistryCredential shown below:
|
"name": "VotingWeb",
"properties": { "description": "VotingWeb front-end service", "osType": "Windows", "codePackages": [ { "name": "VotingWeb", "image": " [parameters('web-image')] ", "imageRegistryCredential" : { "server" : "[parameters('registry-server')]" , "username" : "[parameters('registry-username')]" , "password" : "[parameters('registry-password')]" }, |
Now click the Commit button at the top of the editor and then click commit on the dialog box to save the changes.
The template is now parameterized and ready to use in a build pipeline.
Create a build pipeline
The build has the responsibility to take the source and create the artifacts needed for deployment. To create a new build pipeline, select the Pipelines -> Builds and then the New pipeline button.
On the New pipeline view, click the Use the visual designer link.
Verify that the information is correct on the Select a source view and click Continue .
On the Select a template view, select Docker container and click Apply.
On the Pipeline view, make sure that Hosted VS2017 is selected for the agent.
Clone both the Build an image task and Push an Image task by right-clicking and select Clone task(s) .
Then use the Add a task button to add Copy Files to and Publish Artifact tasks to the pipeline.
Click the first Build an image task and change the settings to match the following (using your Azure Subscription and Azure Container Registry):
On the first Push an image task, change the settings to push the votingweb image to your ACR.
Repeat the same steps for the second Build an image and Push an image tasks.
The Build an image task for the votingdata image should look as follows:
Push an image for the votingdata image should look as follows:
Modify the Copy Files task to look like the following:
Verify the Publish Build Artifacts are configured as follows:
Select the Triggers tab and check Enable continuous integration .
Click the Save button in the top left corner and then again on the dialog to save the build.
Create a release pipeline
The responsibility of the release is to take the artifacts the build created and deploy to an environment.
In the left menu, select Releases and the New pipeline button.
On the Select a template view, click the
Empty job
link.
Now click the Add an artifact button, to configure where the artifacts are coming from.
Modify the settings to match the following and click Add :
Once the artifact is added, click the Continuous deployment trigger (lightning bolt) and enable the trigger.
This enables the release to start when a build completes.
Next, we need to add the variables to pass into the template. Remember all those parameters we added?
Click the Variables tab and use the Add button to create and populate the variables with your values.
When you are done, it should look like following:
The registry password, username, and server are for your ACR and can be retrieved from the Access keys area of your Azure Container Registry in the Azure portal.
We are now ready to create the release task.
Go to the Tasks tab and search for resource . Find the Azure Resource Group Deployment and click Add .
Modify the settings to reflect the following:
Click
Save
in the upper right corner and
OK
on the dialog to save the release pipeline.
Trigger the CI/CD pipeline
Now all we need to do is make a change to the repo and the CI/CD process will start.
In the left menu, select Repos and Files , then click the README.md and the Edit button.
Make some small change and click the Commit button and Commit on the dialog.
Now go to the Pipelines and Builds to see your build status.
Once the build has succeeded, you will need to get the IP address of your service. Currently, the easiest way to do that is in the cloud shell.
- Go to https://shell.azure.com .
- Choose the subscription to which you deployed your Azure Service Fabric Mesh application.
- Add the mesh extension by typing the following command:
| az extension add –name mesh |
- Get a list of the networks and their Public IP addresses, with the following command:
| az mesh network list |
Find the Public IP address for your application and open it in a browser. You should see a page like the following:
Summary
In this article, we’ve looked at the challenges a microservices architecture has, introduced the new Azure Service Fabric Mesh microservices platform, and explored how to build and deploy an application to Service Fabric Mesh. Now that you have a beginning CI/CD pipeline set up for a Service Fabric Mesh application, you can start to explore how easy it is to host your microservices with it. You can find more information on the documentation site .
To contact Jason find him on Twitter @haleyjason