Following up on my recent article on the hidden costs of cloud, I realize that there is far more discuss (https://www.linkedin.com/pulse/hidden-cost-clouds-rarely-discussed-tony-sykes).

Everything appears to be called cloud nowadays and it’s easy to become lost in the hype and find yourself feeling compelled to join the ‘go cloud’ race. Unfortunately, marketeers are making the situation cloudy (sorry, couldn’t resist) through abusing the term by lifting on the positive attributes, without being totally honest about the potential hazards, considerations or limitations involved.

Don’t get me wrong, I am a supporter and believer in the development of technology and ‘cloud’ based solutions, but I see too many people using buzzwords to make business changes without a compelling business case or strategy to support adopting the latest cloud option.

Having designed and built a multi-tenant, pay as you grow, hosted e-mail service back in 2007 (I guess some would call it a cloud now) and then having contracted and supported transformation to both public and private clouds over the past 12 years, I have seen a lot of success, but also plenty of failure to achieve the expected benefits. There are a lot of ‘gotcha’s’ when using cloud services to fulfill all of your IT needs.

Any quick search on the internet is going to support your wildest cloud dreams of:

• Increased flexibility. Scaling to meet your needs, anywhere, anytime
• Faster time to market for applications
• Collaboration across teams
• Maximum data security and disaster resilience
• Pay only for what you use
• Online service dashboards
• The latest technology at your fingertips

All of the above are enabling you to focus on your core business! And that is what you want…right? 

Now, there are a lot of cloud solutions on the market and it is impossible to address them all, or even the tip of the iceberg, but there are some fundamentals that apply to pretty much all of them. Let’s examine those benefits of cloud in more detail so that you can judge if you really need them, have considered them, and if they are likely to achieve your needs.

Flexibility: Modern software enables the decoupling of hardware limitations and solutions/applications have become more modular in design to enable them to scale both up and down. The key question here, is ‘do you need it?’. Examples such as financial systems (month/quarter end processing), web shops (sale periods, Xmas) or batch processing are often used to demonstrate the need to cope with peaks in demand. Traditionally, developers of such environments requested an infrastructure that could sustain the largest foreseeable demand, resulting in over-sized environment being under-utilized 95% of the time. This was largely resolved when virtualization became mainstream technology, and cloud is simply the next step in mass virtualization. 

Whilst cloud offerings play on the fear of under-utilization and the ability to expand to meet sudden growth, many systems are stable and have small to no peaks at all. Although a cloud environment may bring other benefits to stable environments, these can still be achieved without taking the ‘elastic’ option which will inherently have a higher price tag for a stable environment. Knowing the flexibility needs of your individual departments, rather than a generic business model, is essential for choosing the right cloud solution.

Flexibility is also often linked to the ‘anytime, anywhere’ aspect of cloud since the cloud is typically internet based (yes, you can argue if an on-premise environment is truly a cloud in terms of definition). Although true for many countries, internet connectivity/bandwidth globally is still not sufficient to host everything in the cloud and the speed of light didn’t get any quicker in recent years! Having seen the obstacle of virtual workplaces not being able to function/integrate correctly with business applications numerous times because both were delivered from different cloud environments (or on-premise combined with public cloud), it’s important to consider if your multi-cloud strategy is going to still achieve the performance needed across the spectrum of tools that your business needs.

Faster time to market: With infrastructure deployment times enormously reduced (hours in place of what used to take weeks (days if hardware was available)), there is a clear improvement in potential time to market. Not only time is reduced, but the required CAPEX for the necessary hardware can be avoided too. Whilst infrastructure deployment obviously doesn’t itself provide a faster deployment of the application, the maturity of DEVOPS processes and enabling tools in the cloud have delivered significant steps forwards for the application layer. Clouds also boast the latest and greatest technology/versions of products. 

For greenfield/startup situations, all of the above makes perfect sense, but considering that most enterprises have a legacy environment to maintain and develop, moving these to the cloud either requires significant redevelopment or containerization/encapsulation (to comply to the version restrictions of the destination) which can then negate the potential savings through additional licensing and increased operating costs. There is also a danger that the frequency of updates to your applications become obligatory as the cloud continues to develop/deploy new versions whilst your application may not require functional development at the same pace. 

The agility offered by the rapid deployment also presents a new issue in terms of software asset management. Those software assets which are not part of the cloud itself but used by the application layer (in the case of IaaS & PaaS) need to be tracked and reported on to ensure compliance. In the worst case, you may also find that your software is not supported by the cloud of choice or that you become obliged to procure additional software assurance when operating the license in the cloud environment.

Data security: Whilst the increasing levels of distributed denial of service attacks (DDOS) are often in the news, these are not unique to cloud environments and any internet facing environment can fall victim. The same applies to malicious users, data transmission or ‘man in the middle’ (MITM) attacks. Issues which are more specific to cloud based data are those concerning data location and identity & access management (IAM). 

Due to increasing legislation on data protection, cloud service providers have improved the visibility and transparency of the location of customer data in the past years, opening more regional/local data centers to accommodate demands, but IAM is seldom mentioned in cloud sales pitches or noted as part of a business case proposal.

Whilst it’s still prudent to validate the security tools and controls being offered, the main concern today is ensuring that data resides in permissible locations and is not unknowingly transported outside of these locations. For instance, validating which customer data is transmitted through incident management/alerting tooling (internally with the providers support teams) may sound far-fetched, but it is surprising how suppliers unwittingly transmit essential parts of data around the globe, placing their customers in jeopardy of breaching data protection legislation. 

Also, consider who has access to your data, even if it’s located in a permissible and ‘safe’ location. Your IAM controls may have locked down your own access, but your supplier has back-end access to administer the environment. Although this access is normally restricted to avoid administrators being able to view data, their privileges allows them to take ownership of the data, should they try to. Some providers now offer encryption services to avoid this, whereby the customer retains the decryption keys, but ideally, the best way to prevent this is to change your architecture to initiate encryption at the application or database level and retain capabilities in this area. Don’t be fooled by some clouds which offer TLS & SSL encryption as this is only encrypting the data as it is transmitted. The data at rest in the data center often (as much as 90%) remains un-encrypted.

Before choosing your cloud, you also need to determine how you’re going to avoid losing that Single Sign On capability that your users really need. Considering almost all enterprises have adopted Microsoft’s Active Directory and use this as their core identity platform, you need to review the integration and federation abilities of the cloud provider. Whilst most of the major players offer integration through additional connectors, read the fine print. Even AWS states that “AD Connector cannot be used with your custom applications, as it is only used for secure AWS integration for …. Custom applications relying on your on-premises Active Directory should communicate with your domain controllers directly”. You can of course choose to use a Directory-as-a-Service offering whereby yet another cloud provider integrates and manages all federation of your IAM, but this is yet another cost to consider as this is typically a per user/per month subscription fee (your security officer is unlikely to be impressed by outsourcing your IAM).

Personally, the biggest delays in cloud migration that I have seen have been caused by the complexity of achieving correctly implemented IAM solutions.

Disaster resilience: Of course, one of the main reasons to use cloud is its resilience. Nobody enjoys the expense of maintaining under-utilized additional DR hardware or storing extra copies of data and cloud providers typically have extensive continuity plans in place. A basic control is clearly to read through the committed KPI’s to see when you may be back in business if the worst happens (search the internet for Cloud datacenter outages and you will find they happen more often than you may like) or if your environment is sufficiently redundant that no single DC outage can take you off-line. Unfortunately, basics are never enough if you run a large enterprise. Your cloud provider will likely have plans to bring all of the systems back on-line and restore the data to the last available backup (if lost) and that is often where their responsibility ends. But this could actually do more harm than good. If you have transactional systems or a tiered application architecture, you don’t want the front-end website coming on-line before the back-end database has been restored! You also don’t want your transaction system to restart processing when data is restored from yesterday. Traditional DR or BCP plans within a company deal with the sequencing of events to ensure a smooth recovery. This is something that most cloud providers simply ignore because it is impractical for them to do this across all customers. Investigating the obligations and options to conduct sequenced recovery is going to be vital to your recovery needs.

You will likely diversify your cloud usage by adopting a multi-cloud approach, using niche players for some SaaS solutions (e.g. HR, Payroll) and larger suppliers for the bulk of IaaS/PaaS requirements (e.g. web hosting, application hosting) . Whilst this also helps in being more resilient (the chance of all providers going off-line due to a disaster is very small) remember that niche players may have less resilience and all providers may still be integrating into your single IAM environment! If the IAM environment goes off-line, everything is unavailable.

Pay-per-use: This has been one of the biggest selling points of cloud for obvious reasons. That said, traditional outsourcing can also offer this option and is often more commercially attractive than public cloud when utilization can be correctly forecast and remains stable. The biggest ‘gotcha’ lies in the number and type of parameters and the level of granularity. It’s not easy to compare cloud providers because they choose to use different parameters. Whilst one provider delivers granular billing at the CPU/memory level, combined with GB of storage used another will roll this together into a virtual server price with soft limits on storage. Just as you then think to be able to make a calculation for your situation, you find that one provider measures the allocated/provisioned storage whilst the other measures the utilized storage! Knowing which parameters that you can influence through your own usage will help to determine the right cloud provider for you. 

Unfortunately, there is a high chance that billing is not high on the selection criteria when choosing the cloud. This is similar to the traditional capacity management and CMDB that most companies only see as important once they have deployed an environment and realize that they have no visibility on its use or remaining capacity. A major mistake is thinking that capacity management goes away if you move to the cloud! You actually need to take more care in implementing a method to validate your utilization (to enable validation of the bill) and attribute these costs and utilization across the business.

It goes without saying that project management approaches also need to change when adopting cloud-based services. Test/development environments which were traditionally built weeks in advance and over-sized production environments (to buffer the peak load) all need to be avoided in order to avoid the additional costs incurred by pay-per-use. 

Ultimately, you need to realize that pay-per-use is nothing more than a shift in financial risk, combined with the potential for increased volume efficiencies (assuming the provider has better buying power than you do). This often means that a non-fluctuating demand will result in higher long-term costs than a traditional CAPEX based model.

Service dashboards/visibility: Cloud(s) has always been the architect’s playground and unfortunately, only a token gesture was initially made towards the service managers responsible for managing the service in the form of online availability dashboards. Fortunately, things are changing, with the larger IaaS/PaaS players now creating process frameworks and integrating tools (currently the focus being on ServiceNow) to enable the broad spectrum of ITSM to be applied. This needs to be a major consideration if you have an enterprise of any considerable size or are tempted to go to a multi-cloud topology. Most of my previous engagements have always jumped on the technology/cloud train first and then spent years catching up with the ITSM processes and tools causing a lack of visibility in crucial areas such as lifecycle management and configuration management.

Currency of technology: No more end-of-life or end-of-support drama for your operating system. Always the latest security patches and hardware that gives the highest performance possible! What a promise! Now back to reality. Running a business and the corresponding application portfolio to sustain that business, whether it be a website, database, repository, or whatever is about ensuring performance and stability. Of course, you want a supported environment, but most legacy system dramas that I have seen ran into end-of-support issues because the application layer was not compatible with the new OS release. Transformation programs now use containers/encapsulation to keep the necessary (and potentially old) API’s/DLL’s separate from the cloud OS when migrating legacy applications. In such cases, you don’t need the latest and greatest. Similarly, patch management in large enterprises is typically focused on ensuring that patches are deployed within a month (same as cloud) but have exception rules for those applications that will fail if the patch is deployed. Will your cloud supplier allow you to be exempt from a particular patch cycle, or will you be forced to find a new home for your application? You can quickly find yourself in a position where the cloud provider is no longer facilitating you, but you are needing to transform to comply with the cloud provider’s standards.

On hardware, let’s be realistic. No cloud provider is going to refresh their hardware annually to give you the latest chip set or replace all of their storage for newest SSD high performance disks. Like any other business, they have lifecycles and investments to depreciate. Some providers even build their clouds using self-contained, maintenance free data center containers (like ship cargo containers) whereby the hardware is redundant and simply run until a percentage has failed, after which the entire container is scrapped and replaced. If you’re buying cloud based on CPU, RAM and Storage then check the performance KPI’s being offered and lifecycle agreement to ensure that you’re not going to find yourself running on 5-year-old hardware.

Also consider that the majority of clouds are built to satisfy a generic demand. If you have a specific high-performance application that demands terabytes or petabytes of solid-state disk to run your big data analytics, be very specific on these criteria when choosing the cloud provider. I have experienced various business units withdraw from group wide cloud programs because the one-size fits all approach to cloud deploying simply couldn’t satisfy their needs.

In summary:

• Choose the cloud solution at the right level (IaaS, PaaS, SaaS, XaaS) and right level of flexibility to meet the individual needs of the business unit/application rather than adopting a generic business/group driven cloud adoption strategy.
• Consider the entire portfolio of IT needs and the interaction between multi-cloud and legacy environments to prevent performance issues arising.
• Check the obligations for version currency and determine if your application fits the cloud solution.
• Validate the impact of moving to cloud on your software licensing and how you will ensure compatibility and compliance in future.
• Ensure transparency on where your data will reside and if/how it can be encrypted.
• Avoid delays and disgruntled employees by thinking about identity management first.
• Recovering infrastructure does not recover functionality of an application or ensure data integrity! Don’t rely on your cloud(s) to replace your obligations in terms of recoverability.
• Check for weakest links in your DR scenarios such as IAM.
• Calculate (using a use case) the true cost of pay-per-use and understand the variables and parameters that you can leverage.
• Consider ITSM tooling integration and processes alongside the technological aspects of cloud adoption.
• Cloud is not magical, it’s a business model that some of your environment may not fit into. If you’re starting from scratch, then design for cloud, but if you have a legacy, choose wisely and accept more diversity.
• Cloud offers new options in how to deliver technology to support your business, it’s not an obligation.

There are many more aspects of cloud to consider, but these tend to become more specific depending upon the type of cloud platform. I hope to have provoked some of the readers who made it the end of my ramblings and look forwards to your responses.