Microsoft to acquire Deis to help companies innovate with containers
Containers have been at the forefront of cloud transformation in recent years, and for good reason: Container technologies let organizations more easily build, deploy and move applications to and from the cloud. With this increase in agility and portability, containers are helping to make applications the new currency in the cloud. At Microsoft, we’ve seen explosive growth in both interest and deployment of containerized workloads on Azure, and we’re committed to ensuring Azure is the best place to run them. For more, read Scott Guthrie's https://blogs.microsoft.com/blog/2017/04/10/microsoft-acquire-deis-help-companies-innovate-containers/#rKOlAI7rYRutQ1Zj.99Introducing AKS (managed Kubernetes) and Azure Container Registry improvements
Today, we are proud to announce the preview of https://azure.microsoft.com/en-us/services/container-service/our new managed Kubernetes service. We have seen customers fall in love with our current Kubernetes support on Azure Container Service, currently known as ACS, which has grown 300% in the last six months. Now with the preview of AKS, we are making it even easier to manage and operate your Kubernetes environments, all without sacrificing portability. This new service features an Azure-hosted control plane, automated upgrades, self-healing, easy scaling, and a simple user experience for both developers and cluster operators. With AKS, customers get the benefit of open source Kubernetes without complexity and operational overhead. Read about it in the https://azure.microsoft.com/en-us/blog/introducing-azure-container-service-aks-managed-kubernetes-and-azure-container-registry-geo-replication/.Creating Reliability Through Chaos With Azure VMs and Gremlin
Creating Reliability Through Chaos With Azure VMs and Gremlin The idea of “Chaos Engineering” isn’t just about putting faith in a provider to stay online, it’s finding ways to simulate failure in order to determine that you’ll withstand an outage of any kind within your application. This means that if a number of your app servers take on a large portion of traffic and are highly CPU taxes, you’ll know how to properly scale your application to withstand it. If portions of your application infrastructure were to take on a massive amount of packet loss, how does your team respond? Chaos engineering helps answer some of these questions by allowing you to simulate the possibilities of what a failure may look like in your production environment. For some, using tools like Chaos Monkey has helps produce load and service failures to help create attack simulations. Lately I have been working with Gremlin, which acts as a “Chaos-as-a-Service” through a simple client-server model. Read more here on medium
Azure AKS Security Hardening
Hello Folks !! I am back with a new blog . This time I will try give a brief overview about Azure AKS Security and Baseline. Lets gooo !!!! What is Azure AKS - Azure Kubernetes Service (AKS) offers the quickest way to start developing and deploying cloud-native apps in Azure, datacenters, or at the edge with built-in code-to-cloud pipelines and guardrails. It is mostly used as a scalable platforms nowadays. Current Application requirement includes the scaling , performing and most importantly zero downtime , which is covered by AKS service of Azure. Containerization of any application in AKS is the best way to reduce downtime and cost optimization of your infrastructure. AKS features and benefits The primary benefits of AKS are flexibility, automation and reduced management overhead for administrators and developers. For example, AKS automatically configures all of the Kubernetes nodes that control and manage the worker nodes during the deployment process and handles a range of other tasks, including Azure Active Directory (AD ) integration, connections to monitoring services and configuration of advanced networking features such as HTTP application routing. Users can monitor a cluster directly or view all clusters with Azure Monitor. Now having a brief overview of Azure AKS , lets move on Azure security features or we can call it as Azure Baseline for security of AKS , that it offer's - Security related to AKS Related to Networking - By default, a network security group and route table are automatically created with the creation of a Microsoft Azure Kubernetes Service (AKS) cluster. AKS automatically modifies network security groups for appropriate traffic flow as services are created with load balancers, port mappings, or ingress routes. Use AKS network policies to limit network traffic by defining rules for ingress and egress traffic between Linux pods in a cluster based on choice of namespaces and label selectors. Networking allows the filtering of traffic to not only to AKS but also entering it to current infrastructure. Since mentioned about the namespaces in AKS , it is a whole virtual environment that is separated within Kubernetes cluster , we can configure alert based networking rules for particular namespace also. 2) Using the traditional method ( i.e. auth from AD or role creation) for AKS - Kubernetes includes security components, such as pods , and nodes security . Meanwhile, Azure includes components like Active Directory, Azure Policy, Azure Key Vault, and orchestrated cluster upgrades. AKS combines these security components to: Provide a complete Authentication and Authorization story. Leverage AKS Built-in Azure Policy to secure your applications. Authenticating with the password and keys for developers using Azure key vault .Setting up Azure policy like conditional access policy for better security for Azure updates. 3) Using Azure Application Gateway and WAF - Use an Azure Application Gateway enabled Web Application Firewall (WAF) in front of an AKS cluster to provide an additional layer of security by filtering the incoming traffic to your web applications. Web Application firewall uses a set of rules for filtering out the traffic , which we will get injected into your cluster or nodes. Also Application gateway act as proxy for all the traffic , you can also configure route table for routing of the traffic , when the traffic injects inside the application gateway. Application gateway also provides an external IP , which helps to not expose our main ip in which our application or pods are running. Also using an API gateway for authentication, authorization, and monitoring for APIs used in your AKS environment. It acts as a front door to the microservices, , and decreases the complexity of your microservices by removing the burden of handling cross cutting concerns. 4) Configure central security log management - Enable audit logs from Azure Kubernetes Services (AKS) master components, kube-apiserver and kube-controller-manager, which are provided as a managed service. kube-auditaksService: The display name in audit log for the control plane operation masterclient: This is the display name in audit log for MasterClientCertificate, the certificate that you get from aks get-credentials node client: The display name for Client Certificate, which is used by agent nodes. You can also export these logs to Log Analytics . Use Log Analytics workspaces to query and perform analytics. Use Azure blob storage for storing of the logs and archiving them with various tiers options in Azure. 5) Locations approving in Azure - Use Conditional Access Named Locations to allow access to Azure Kubernetes Service (AKS) clusters from only specific logical groupings of IP address ranges or countries/regions. This requires integrated authentication for AKS with Azure Active Directory (Azure AD). Limit the access to the AKS API server from a limited set of IP address ranges, as it receives requests to perform actions in the cluster to create resources or scale the number of nodes. If you want to know how you can configure this named locations , you can go to this Azure link - https://learn.microsoft.com/en-us/azure/active-directory/reports-monitoring/quickstart-configure-named-locations 6) Isolate the system which are storing data - Logically isolate teams and workloads in the same cluster with Azure Kubernetes Service (AKS) to provide the least number of privileges, scoped to the resources required by each team. Use the namespace in Kubernetes to create a logical isolation boundary. You can also implement separate subscriptions or working directory of the AKS cluster , which are containing the pods with sensitive information or any type of Database, which are prone to attacks. 7) Encryption of all the sensitive information - It is always good to encrypt our data that is exposable to internet in HTTPS. You can create an HTTPS ingress and use your own TLS certificates for your Azure Kubernetes Service (AKS) deployments. Kubernetes egress traffic is encrypted over HTTPS/TLS by default. You can review any potentially un-encrypted egress traffic from your AKS instances. This may include NTP traffic, DNS traffic, HTTP traffic for retrieving updates in some cases. Here are some of the methods , for hardening and maintaining your AKS cluster security. There are also many third party applications which you can integrate with your AKS cluster , but I will recommend to you use them wisely . Go through there files and changes that they will make to your cluster. Thanks !!!!!Kubernetes now Generally Available on Azure Container Service
We announced preview support for Kubernetes in November 2016. Since then, we have received a lot of valuable feedback from customers. Based on this feedback we have improved Kubernetes support and now move it to GA. With today’s https://azure.microsoft.com/en-us/blog/kubernetes-now-generally-available-on-azure-container-service/, we again deliver on our goal of providing our customers the choice of open-source orchestrators and tooling that simplifies the deployment of container based applications in the cloud. The ACS team are announcing today our next wave of features that includes: Kubernetes now generally available (GA) – We announced preview support for Kubernetes in November 2016. Since then, we have received a lot of valuable feedback from customers. Based on this feedback we have improved Kubernetes support and now move it to GA. Preview of Windows Server Containers with Kubernetes – Coinciding with latest Kubernetes release, this is a great time to provide additional choice in orchestrator for Windows Server customers using ACS. Customers can now preview both Docker Swarm (launched in preview last year) as well as Kubernetes though ACS, providing choice as well as consistency with two of the top three Linux container orchestration platforms. DC/OS 1.8.8 update – We are updating our DC/OS support to version 1.8.8. DC/OS is a production-proven platform that elastically powers both containers and big data services. For the full announcement and more details, check out Saurya' https://azure.microsoft.com/en-us/blog/kubernetes-now-generally-available-on-azure-container-service/for the announcement. We love hearing from our customers about how they are using containers on Azure and the benefits it brings to their application development lifecycle. We hope to hear from you, too.
What AKS node types support AVX2?
Title. I have an AKS cluster deployed, and one of the scripts within a running pod fails because of what seems like an unsupported instruction set (AVX2). I looked into documentation and used the az cli, but couldn't find detail of what each image or series support. I could run multiple nodes and check manually, though there must be a better way. Right?
MPI job in cluster built with Standard D2s v3 (2 vcpus, 8 GB memory)
Hello everyone, I am new to Azure. And I tried to build a cluster with Ubuntu nodes in Azure. So I chose D2s v3 (2 vCPUs, 8 GB memory) and Ubuntu1604 Image to build tow VMs as the master node and worker node. I just followed the guidebook to create these two VMs and established /etc/hosts in each node and set up NFS so that the /home of the master node can be mounted on /home of the worker node. It worked, I can passwordless ssh to each node from the other. I ping node001 from the master node: miker2746@mt-test21:~/rm-test1$ miker2746@mt-test21:~/rm-test1$ ping node001 PING node001 (10.0.0.25) 56(84) bytes of data. 64 bytes from node001 (10.0.0.25): icmp_seq=1 ttl=64 time=0.963 ms 64 bytes from node001 (10.0.0.25): icmp_seq=2 ttl=64 time=0.553 ms 64 bytes from node001 (10.0.0.25): icmp_seq=3 ttl=64 time=0.622 ms 64 bytes from node001 (10.0.0.25): icmp_seq=4 ttl=64 time=0.748 ms 64 bytes from node001 (10.0.0.25): icmp_seq=5 ttl=64 time=0.709 ms ^C --- node001 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4057ms rtt min/avg/max/mdev = 0.553/0.719/0.963/0.139 ms miker2746@mt-test21:~/rm-test1$ But when I ran my MPI job in this cluster, it failed and showed some errors as follows: miker2746@mt-test21:~/rm-test1$ mpiexec -n 2 -f ~/hostfile fds random_meshes.fds ^C[mpiexec@mt-test21] Sending Ctrl-C to processes as requested [mpiexec@mt-test21] Press Ctrl-C again to force abort [mpiexec@mt-test21] HYDU_sock_write (../../utils/sock/sock.c:418): write error (Bad file descriptor) [mpiexec@mt-test21] HYD_pmcd_pmiserv_send_signal (../../pm/pmiserv/pmiserv_cb.c:252): unable to write data to proxy [mpiexec@mt-test21] ui_cmd_cb (../../pm/pmiserv/pmiserv_pmci.c:174): unable to send signal downstream [mpiexec@mt-test21] HYDT_dmxu_poll_wait_for_event (../../tools/demux/demux_poll.c:76): callback returned error status [mpiexec@mt-test21] HYD_pmci_wait_for_completion (../../pm/pmiserv/pmiserv_pmci.c:501): error waiting for event [mpiexec@mt-test21] main (../../ui/mpich/mpiexec.c:1147): process manager error waiting for completion miker2746@mt-test21:~/rm-test1$ I have built exactly the same cluster with similar VM size on AWS EC2 and Google Compute Engine before, they all worked very well and never showed these errors before. I tried to find the answers and solutions to my problem by reading documents of Azure. And according to https://docs.microsoft.com/en-us/azure/virtual-machines/linux/sizes-hpc, it seems that only VM size of H16r, H16mr, A8 or A9 is allowed to run MPI jobs. Is that true? Or it just means that MPI jobs can experience low latency RDMA when they run in HPC cluster built with one of these 4 VM sizes. So my questions are: 1. Can anyone tell me what's wrong with my cluster, why will I have this kind of errors. Is that because I haven't set up Virtual Network correctly? 2. Can I run MPI jobs on a cluster that built with VM sizes other than these 4? 3. If MPI jobs only allow running in these 4 VMs, how can I set up a cluster with this particular VM? Thank you very much Michael
Interactive containers in Azure
Hello All, I'm trying to figure out how to run an interactive container in azure. What I would do for a local container on docker: docker run -t -i dockerID /bin/bash Now i can create the container on azure, but if it doesn't have an active back ground process, it just exits immediately. I could probably just create a shell script that keeps the container alive in the background and then use az container exec to get a bash shell, but it seems wasteful to leave the container running all the time and not the right way to do things in container land from understanding. So... what is the right way to create an interactive container in azure (like this one for example https://github.com/brimstone/docker-kali)? ?
