In Customer Technology Preview 1 (CTP1) we introduced a small group of customers and partners to a new and important feature segment being released with Microsoft System Center 2012 SP1: Service Provider Foundation (SPF). The CTP1 release provided enough of a preview of SPF to clearly demonstrate its role as the web API for accessing the management capabilities of System Center. SPF is one of those great engineering investments we can make as a product team. It is a response to feedback from existing customers and partners who have built impressive private clouds without the benefit of comprehensive programmability for System Center. SPF also increases strategic emphasis on making Windows Server + System Center an accelerant for hosted infrastructure and service businesses that are growing very rapidly. CTP1 showed how we designed and built SPF to meet the requirements and preferences of both developers (a consistent programming experience) and IT professionals (new capabilities *and* compatibility with existing management systems investments). Now, as more customers and partners are getting their hands on CTP2 and the latest updates to SPF and the rest of System Center, the power for managing multi-tenant environments becomes (more) evident. But first…
“How SPF Helps Service Providers” or “Elevator Pitch on SPF Value Prop”
We are introducing SPF as the API for System Center to unlock System Center’s breadth of management capabilities without the need for an all-or-nothing value proposition. Service provider feedback was succinct and specific, and SPF v1 enables all of the following requirements we heard:
· Let us use our existing customer-facing UI
· We need web APIs to enable hosted IaaS portals
· System Center needs to support multi-tenancy
· Give us an API that spans instances of System Center
· Please remove the need for us to manage tenant networks
· Give us a way to customize and extend our existing offerings to differentiate
· Help us offer value-add services like monitoring and backup
One scenario I’ve heard several times in the last few months is the case of a successful hosting business being asked more and more by enterprise customers to offer hosted forms of the Microsoft collaboration products (Exchange, SharePoint, and Lync). With little or no impact to existing operations initially, these service providers are looking at System Center w/SPF as a strong set of capabilities for managing this new offering. In the process, new service offerings and self-service user paradigms can accrue value to their existing business.
Even more common is the ability for System Center w/SPF to address a specific pain point for service provider operations. To highlight one item from the above requirements we heard, many of our customers and SPF early adopters report spending huge amounts of time managing tenant networks and IP addresses. By exposing SPF management of VM networks on Windows Server 2012 these service providers can essentially delegate this commodity task to self-service users directly, which in turn allows the service providers to focus more on new services that differentiate from their competition.
“A Quick Recap of CTP1” or “Brief Overview of SPF Internals”
The following is one of a few different graphical depictions we’ve used for describing “what’s in SPF?”
The picture is a bit of a teaser since this post isn’t about SPF Extensibility beyond what is shown in the above figure and a few quick comments here. Extensibility will make its formal debut in the Beta release. Specifically, the piece in the above diagram that says “defined extension points” becomes ready for use in beta and remains hidden until then (though some of the more adventurous adopters have gotten an early start on SPF extensibility by jumping into the effort to “create a new resource” which is also shown in the above figure). For immediate purposes the focus here is on the inset box that encloses the parts of SPF, and specifically three main pieces of the service: resource model (definition of objects, or resources, that can be managed, like a VM); action mapping (pointers to the implementation for actions that can be taken on the resources, like create, start, or stop); and task execution (the implementation, like PowerShell or System Center Orchestrator runbooks).
Anyone who installed and experimented with CTP1 would have experienced some or all of the following:
· SPF has a standalone setup experience and no database
· SPF is built as a RESTful web service running under IIS on Windows Server (CTP1 was Windows Server 2008 R2 only, CTP2 is Windows Server 2012 only, Beta onward will support either)
· SPF implements the Odata protocol (CTP1 was Odata v2, CTP2 is v3RP) for operating on collections of objects, where URL/path directs access to the right objects and Odata query syntax controls the filtering by attribute (essentially “SELECT/UPDATE path/object WHERE filter” in SQL terms). Using Internet Explorer to do a sample GET action on the clouds associated with a particular VMM server looks like this:
http://<SPFserver>:8090/SC2012/VMM/Microsoft.Management.oData.svc/Clouds?$filter=VMMServer eq ‘<VMM server name>'
· SPF exposes capabilities of Virtual Machine Manager (VMM) and Data Protection Manager (DPM) 2012 RTM version, in support of the hosted IaaS theme:
o VMM functionality is obviously central to the IaaS theme, and CTP1 focused on the creation and operation of VMs (management of templates, etc.) as IaaS step #1.
o DPM support in CTP1 targets the basics of creating and joining data protection groups for guest VMs created through the VMM functionality exposed.
· It is possible and relatively easy to find (under the inetpub vroot) those SPF implementation artifacts described above (resources as *.MOF, mapping as *.XML, implementation as *.PS1) and modify them (and then, hopefully, think “I probably should put that back the way it was in case I broke something…”)
As I mentioned in my introduction, CTP1 was the first release where developers and IT Pros could look at what we’d built and assess how well we met their requirements and preferences. In this regard, exposing System Center capabilities through a RESTful web service was more than just trendy, it builds consistency with other assets (e.g. the RESTful web service for System Center Orchestrator) and with modern styles for portal code built to the cloud design point. And simply by the fact of producing a RESTful API for System Center, and thereby strengthening System Center’s family of automation and integration features, we show IT Pros that we’re committed to adding value with new and complementary management capabilities which can integrate with their existing assets.
“SPF Comes on Strong in CTP2” or “Redmond, We Have a Database”
With the foundation laid by CTP1, while our Technology Adoption Program (TAP) members have been kicking off their proofs-of-concepts, prototypes and pilots, the SPF engineering team has been busy enabling the parts of SPF that really demonstrate its incremental value beyond just providing programmatic access to System Center. That value comes from two main categories of enhancements in SPF CTP2: aspects of multi-tenancy, security and self-service (tenants & roles/workflow); managing and abstracting the physical world (scale unit & aggregation across). We’ll go into more detail on these in a moment, but the first change to highlight in CTP2 is that SPF now has a database (SCSPFDB) where we can store metadata associated with the enhancements I just described. In CTP2 the SQL Server must be the default SQL instance and not a SQL named instance, though Beta will change that to allow either. Anticipating both scaled out and highly available deployments, it is possible to install SPF multiple times as someone would do for building out an IIS web farm, while pointing to an existing database when a prior SPF setup has already created it. And we will be performing tests before SP1 RTM to ensure compatibility with the scale and high availability mechanisms commonly used by IIS (e.g. load balancers) and SQL Server (e.g. replication) in conjunction with Windows Server Failover Clustering.
The next thing to talk about is multi-tenancy, and key personas in the typical workflow that turns bare metal into management experiences for empowered self-service users. The following identifies the main personas and their core tasks as well as the order of execution (when you see the word “stamp” simply translate it to “scale unit” and we’ll come back to that topic in a minute):
Formalizing the notion of “tenant” is obviously central to the broader mission of making System Center a powerful tool for managing multi-tenant environments. SPF materializes the tenant ID and stores it in the SPF database, and as a property on VMs managed through SPF. Roles in the above workflow are simply projections of scope associated with Role-Based Access Control (RBAC). The highest order persona is the hoster admin, who has scope across the entire fabric (multi-datacenter). This persona provisions servers, configures the management footprint templates and prepares offers, then delegates to tenant admin. Tenant admins have scope and perform aggregate actions within the tenancy (e.g. establishing networks required by their applications to be hosted) and further delegate management to the self-service users (SSUs). In CTP2, “tenant admin” is not yet implemented and the CTP2 experience is one where the two types of admin, hoster and tenant, are essentially collapsed into one. The two roles will be fully distinct and functional in the Beta release.
Security for SPF is implemented such that user requests are validated against role metadata stored in the SPF database. Once it is verified that a requestor has access to the scope and specific objects in the request, SPF uses credentials for the underlying service app pool (provided by the IT Pro during SPF installation) to perform management tasks on behalf of the requestor. RBAC is thereby consistent with whatever scope/SPF user roles the admins have allocated to the various personas. In CTP2 SPF supports both claims-based (from secure token services) and Windows authentication (no hard Active Directory dependency), and from Beta onward will also support basic authentication (where requests carry name/password over https).
Getting back to the scale unit, or “stamp” as it’s called in the SPF vernacular, the idea is relatively simple: management functions need to be aware of, but not limited by, maximum scale (regardless what the max is) of the underlying management servers exposed through SPF, specifically VMM. Some people claim the term “stamp” came out of Office 365 operations, but regardless of its origin, the manufacturing metaphor is of something that can be print-replicated from a handful of templates (I tend to think of DVDs that are literally stamped, v. laser-etched, with Microsoft software in our Puerto Rico manufacturing facility). In the case of SPF and System Center the template is of the management footprint. It consists of VMM at its heart, with one or more companion Operations Manager (OM) servers, and as noted the size of this particular scale unit is whatever max capacity (physical hosts and virtualized guests) we support for VMM. The following example topology depicts how the management stamp concept fits in the context of a multi-tenant hosted environment:
As noted in the figure, the current (at the time of this blog post) VMM scale maximum is published as 1,000 host machines and 25,000 VMs. When the multi-tenant hosted environment grows to this point the service provider needs to add another management “stamp” so that tenants can continue to subscribe and expand their subscriptions normally. As they do this, tenant allocations will become distributed and in some cases completely ignore (as they should) the stamp boundaries. Once that happens, SPF’s ability to aggregate management actions across stamps becomes critical (e.g. to see a list of all of a given tenant’s VMs), but also serves as a kind of virtual scale-out for the VMM server. In theory, the largest datacenter footprint (multi-region) in the world could fit in 36 stamps with an even more theoretical single SPF farm providing access to all of it. The actual maximum number of stamps that can be fronted by a single SPF farm is TBD, will be published by RTM of SP1, and is likely to be substantially less than 36 but hopefully in the double digits. SPF aggregation in CTP2 provides access to these objects: clouds; VMs; VM templates; VM services; VM service templates; user roles.
Another important CTP2 enhancement to highlight is the ability of System Center App Controller to manage applications through SPF. This is primarily for the scenario where a tenant who uses App Controller in a private datacenter can use the same tool to also manage applications that run on allocated assets of their tenancy in some hosted environment.
Conclusion & What’s Next
There is a lot of new stuff in CTP2 for adopters to use to expand the scope of experimentation and pre-production development as part of ongoing TAP engagement. The SPF feature set will soon transition to public availability as we lock down the final feature scope for SP1 RTM and shift into fit-n-finish to hit our quality bar for production readiness. From Beta onward adopters can expect upgrade support that will minimize the cost of incremental releases on the road to RTM late this calendar year. It is also during this time that we’ll be adding full multi-language support consistent with Windows Server.
Lastly, the “automation and integration” affinity between SPF and Orchestrator cannot be ignored. At a minimum, Orchestrator is a key piece of the extensibility story for SPF (which we’ll go into in a later posting). From Beta onward, SPF will give up its standalone setup experience and merge with the setup for Orchestrator, and the affinity between these components will grow over time.
On behalf of the SPF team, thank you for your interest in SPF, and we’re very eager for your feedback in whatever form you want to give it.
Download:
|
Location |
Description |
|
http://go.microsoft.com/fwlink/?LinkId=254659 |
The entire installation of System Center 2012 Service Pack 1 CTP2. |
|
http://go.microsoft.com/fwlink/?LinkId=254803 |
Documentation for all the components. |
-John Ballard