In part one, we highlighted Tier 3’s strong showing in a recent performance benchmarking survey.

So what to make of reports like this?  Here are 6 observations that might help you, a buyer of cloud services, interpret findings like this.

Two Things These Reports Can Tell You

  • Quantification of vendor claims & normalization. Vendors will describe their products as “high performance.”  Well, compared to what?  And given that each vendor has a different hypervisor and hardware in their data center – in addition to different instance sizes, or a build to custom specs capability (Tier 3’s approach) – it is hard to gauge clock speeds across providers.  This particular report cuts through that.
  • Each cloud on this list is likely better than another competitor – your internal data center. Every cloud provider on this list is operating at scale, releasing new features often, and supporting a multi-tenant service.  As a result, they are almost certainly delivering better performance than what you have on-premise today.  And due to the expansive self-service offerings some of these providers offer, superior agility relative to your internal data center is a given.  The really interesting – and possibly confusing - thing?  Some of the vendors on this list deliver all of this using hardware that is actually worse than the gear you own and operate today.  That’s why the “feeds and speeds” conversation isn’t really worth having with infrastructure providers.

Two Things These Reports Can’t Tell You

  • The human cost of each cloud service. We all read the endless stream of articles on the cloud price wars.  But the cost of administering public clouds, in the form of IT salary, is far higher than the IaaS spend for more enterprises.  How does each cloud reduce this cost, and help enterprises boost their “VM to admin” ratio?  What kinds of automation can IT take advantage of?  How much time will you need to spend bolting on modules to ensure proper compliance and governance?  How does each cloud help transform IT from gatekeeper to an enabler?  These questions can only be answered by the vendor and their respective ecosystem.  In fact, these answers will play a far larger role in your ultimate cloud success than comparing feeds and speeds.
  • Roadmap & innovation track record.  Ben Kepes hints at this in his article on the study.  Customers want to know who they are hitching their wagon to when they choose a cloud partner.  What are your target workloads?  What are you doing for platform as a service and other application services?  How are you expanding self-service and API access?  What about private and hybrid cloud scenarios?  How are you integrating cloud into other ecosystems and other lines of business?  Buyers need to make sure they are aligned with the vendor’s answers to maximize fit.  Sure, you will eventually be using multiple public clouds down the line.  But that makes the answers all the more critical so you can understand where things will fit in the years ahead.

Two Important Takeaways

  • Performance comes easier on some clouds than others. You can build high performance apps with a great user experience on almost any platform.  Just know that significant human capital may be required, depending on the vendors you choose. Here at Tier 3, we have customers in enterprise IT as well as ISVs who can run legacy applications – even those that aren’t natively multi-tenant – on our cloud with ease.  Why?  We’ve replicated on-premise levels of resiliency.  Our platform delivers HA live migration for VMs and fast VM restart.  And we initiate our own live migration when required.  Enterprise apps that assume infrastructure resiliency are going to run well on our cloud, and do so with less elbow grease of IT and development.
  • Different clouds for different apps.  Kepes also notes that this report is unlikey to stop the flood of customers migrating to AWS and other large clouds.  He is almost certainly right.  Running highly elastic, web-scale applications on a global scale with cheap, destructible VMs is the sweet spot for these providers.  But the architecture offered by these providers doesn’t suit every class of application.  Helping draw these lines is the true value of benchmarking reports like this.