Can you measure IT Service Quality?

I realize that this is a somewhat broad question with a plethora of answers, all potentially being correct. When I started my career in IT some 25 years ago, people criticized the IT sector as being immature. Whilst countless frameworks, policies, processes, regulations, etc. have been implemented to improve the maturity of IT services over the years, I see more service quality issues than ever before. 

It’s also not surprising that IT environments suffer service quality issues where there are typically more components involved than a Boeing 747 (6 million parts for those now tempted to search for the answer) and a huge diversity of software comprising of millions of lines of code, developed by thousands of individuals.

If you don’t have IT Service Quality issues, please stop reading now and contact me so that I can understand how you have achieved this. If you do, please read on to understand what I have experienced to be the key issues and then contact me to share your experiences.

From my experience and akin to non-IT based Services, Service Quality is measured through 5 inter-related characteristics:

  1. A stable foundation (infrastructure): Maintaining a fit for purpose, effective and efficient foundation
  2. Dependability: Keep everything secure and operational
  3. Responsiveness: Resolving incidents quickly or making changes happen on time
  4. Customer focus: Understanding the business and it’s changing needs
  5. Assurance: Confidence that employees have the skills, knowledge and confidence to deliver the work

Whilst dependability, responsiveness and a stable foundation are typically defined, represented and validated through the adoption of architecture, various KPI’s and processes (commonly covered through the adoption of ITIL, TOGAF, Zachman frameworks, etc.), customer focus and assurance characteristics are more elusive in their definition.

The aforementioned characteristics also tend to receive the majority of focus in any contractual relationship with an IT Services supplier, whilst customer focus and assurance only receive attention during the initiation of an engagement. This is also the major differentiator between outsourced and insourced services, where customer focus and assurance tend to receive more attention when IT is internally sourced. That said, many internal IT organizations/departments, even when outsourcing services, have simply taken it upon themselves to retain the responsibility for assuring the necessary quality of service towards the business, indirectly removing the obligation from their respective supplier(s). This often leads to frustration as services degrade or changing business demands are not met with the agility that is expected.

Supplier’s reading this may now feel the urge to reject this statement on Customer focus, maintaining that they perform multiple Customer Satisfaction surveys, C-level discussions, host innovation workshops, train and coach their personnel and all manner of customer focused activities. If so, please provide an example of where you pro-actively understood the change in business needs, developed an alternative KPI to that which you already delivered, trained your team on the business demands and then offered to update your service to fit the needs of your client (without presenting this as a sales opportunity)! In 25 years, I can count the number of times that this has happened on one hand.

Quality assurance when outsourced has traditionally been largely implicit, through brand reputation and holistic annual satisfaction surveys (i.e. Trust us, we’re the best!). Occasionally, suppliers may offer some insight into the composition and competency of their teams, but as previously mentioned, this is typically only prior to contracting or for the duration of a transitional period. Suppliers are very much focused on delivering a service that is subject to quality control (pre-defined KPI’s) rather than maintaining a method by which transparency in resource capabilities is given. The fear here appears to be that a client/customer will try to influence or micro-manage the service beyond the obligatory KPI’s.

From a market perspective, it’s also logical that Service providers in outsourced situations are reluctant to provide quality assurance measures that extend beyond their reputation. After all, their competitors in the SaaS and PaaS cloud services arena are also evaluated purely on their reputation! I’ve never heard a client asking if Amazon or Google can provide an overview of the skills and certificates of their employees before buying the service.

Also, with an increasing interest in Artificial Intelligence and Robotics, increased cloud adoption and re-positioning of traditional service providers as brokers, who or what is responsible for defining, measuring and validating the quality of your IT services?

Assuming people are still interested in ensuring that they receive some measurable level of quality in their Services, it’s time to review your sourcing strategy and supplier landscape to decide who is going to be responsible for each of the 5 characteristics.

The next step is to work backwards through the list of characteristics. Whether outsourced, insourced or multi-sourced, you need to have confidence that what you are going to deliver will be delivered by people that are capable of doing so (even in robotics and AI, you need the confidence that the robot and AI is proven and able to do the task at hand). If market reputation is sufficient for you to be confident, then great, define how you will measure this Service Quality Assurance (SQA) characteristic (e.g. market share, Gartner magic quadrant, etc.) and lock this into your overall quality management system. However, if you’re like most of my clients and need a little something extra to boost confidence levels, consider SQA’s that oblige your supplier (or even your internal team) to report on qualifications achieved, business awareness trainings completed, or a governance structure that reveals planned resource changes for example. This won’t increase service quality, but it will at least provide a measure by which you can identify the source of any lack/decrease in quality and hopefully pre-empt it occurring.

Next, consider the customer focus. Depending on the sourcing relationship, you may not want your service providers engaging directly with the business. Here, you need to consider three things:

  1. The value of an IT service is not in being a facility (like a heating system that keeps the office warm), it’s in being a facilitator (ensuring that the business runs better). If your service feels like a facility, get external help to understand how to turn it into a facilitator.
  2. A business needs to adapt to change and IT services need to adapt with them. Making the change happen on-time and on-budget are key concerns. Ensure you have the people and skills to make this happen (hence the reason to focus on quality assurance first)
  3. Once a change has happened, stable operations are needed. These operations need to reflect the business needs in revised KPI’s and services.

The last item above is the most challenging. This is where the chain of quality metrics is often broken, leaving a gap between the business focus and the IT service, resulting in KPI’s for responsiveness and dependability simply defaulting to the traditional ‘availability’ or ‘incident response’ KPI’s whereas the business is more interested in ensuring that the ‘transaction’ has been completed on time.

Whilst it may be impossible to guarantee a ‘transaction’ based KPI due to external dependencies or inflexible, risk averse supplier behavior, consider at least implementing a method of measurement, even if no consequence of failure is applicable. So many IT services provide measure of availability or capacity that have no tangible connection to the business. The results of a CSAT survey will be significantly easier to discuss if you have been able to track the influence of the IT environment on the business performance, rather than presenting a wonderful watermelon report based on irrelevant availability KPI’s.

With your ideas for responsiveness and dependability now aligned to the business, determine how these may be measured and reported. Synthetic transaction monitoring is a common example, but with advances in CMDB capabilities, it is becoming easier to monitor entire eco-systems and predict failures too.

Lastly, whilst virtualization techniques and redundant architectures have extended the useful lifespan of many IT assets, I regularly encounter ridiculous situations where IT components have been retained beyond the vendor’s end-of-life announcement. If you choose to maintain such an environment, I can only hope that you have a very well-maintained risk register that has highlighted the risks of the approach in your defense. There are sufficient offerings in the marketplace that can: turn a CAPEX into OPEX to remove investment hurdles, provide pay as you grow schemes, offer cloud-based alternatives, etc. Whilst service quality needs to be driven from and aligned to the business perspective, without a reliable foundation, you cannot expect to achieve any of the above. Contrary to the beliefs of many young drivers, you cannot expect to set a new land speed record in a 1990’s Volkswagen Golf!

Please let me know your views and success stories.

One thought on “Can you measure IT Service Quality?

Leave a Reply

Your email address will not be published. Required fields are marked *