“Is It Up?” vs “Does It Work?”

There is a world of a difference between “It’s up” and “It works”. 
I could go on about this for hours, but, it basically boils down to your customers — they care about the quality of the experience, and only the quality of the experience. It may be up as far as you’re concerned, but if interacting with it sux, well, from the customer’s perspective, does it actually work?

 Take the above comic. Your implicit assumption might be something like “It’s a stupid UI” or some such. Much as we’d like to blame everything on the poor folks in UX, it really could be as much about poor performance aspects, you know?
Think of all the times that you’ve hit “Submit”, and gotten back a server error, searched for something with no result when you know it’s there, and so on. You need to make sure that those aspects of the Customer Experience are covered, as follows —

• • Request Rate (aka “Does it scale?”): How many requests can be serviced before the system craps out? This encompasses averages, bursts, and so forth.
• • Success Rate (aka “Are these the responses the customer needs?”): Note that strictly speaking this isn’t the same as Errors, since the customer might be satisfied by a degraded or approximate response.
• • Latency (aka “How quick was the response?”): The time taken to actually service the request, measured from the time the customer dispatched it. Remember, it’s the customer’s view of the world!

A common acronym used for the above is RED (Rate, Errors, Duration), and from an observability perspective, it’s about the things one needs to measure to get an external view of the system, treating the system as a black box. (••)

Mind you, this isn’t just about the system as a whole, it applies to all individual services, which might themselves be customers of other services.Whats more, the metrics apply not just to the individual services but to the activities which might cross-cut the services. For example, if it takes 2 seconds to buy an item, but 2 months for the items to be delivered, well, that may not exactly promote a harmonious customer experience!

Which, funnily enough, brings us back to UX. You can’t define the customer experience without these metrics, and you can’t define these metrics without knowing what the customer experience needs to be. Which shades right into UX…

(•) And yes, the quality of the experience can stuff like “how it’s built” (unboxing videos, device teardown…), “CEO worship” (Elon Musk, Steve Jobs…) but that’s getting way too meta.
(••) Don’t forget that this isn’t just about averages. You should be looking at distributions, percentiles, etc.