UX Metrics for the Modern Web
The Web started as a relatively static medium. Websites had content that was mostly flat and rarely changed. Flash forward 20 years, the Web is filled with super-dynamic sites and applications. All of these applications are dynamic on the client side, as well as the server side. This shift—toward more and more dynamic applications that rely on personalized imagery and updates, customized content and loads of videos—has forced Web performance experts to create new ways to objectively measure the performance of an application or site. An understanding of these metrics is good, not only for Web performance experts and front-end coders but also for CMOs and others who count on good Web performance to close deals, win customers and keep users engaged. In this article, I'll run through some of the basic terms and their relevance in the world of dynamic websites.
Earlier generations of testing tools would simply start a timer, initiate the page load, and then stop the timer after the underlying Internet connection was disused. In the "Web 1.0" world, this was a sufficient test—the browser needed all the content in order to actually render the page and get that user happily "using." On the modern Web "2.0+," pages don't need everything in order to be actually functional. Secondary content and/or personalized content may be loaded asynchronously (for example, below-the-fold loading) even as the user engages with the application. Thus, Internet connection idleness is no reflection of user experience, and "fully loaded" has become less relevant.
The "document complete" event is fired in the browser when the document load is complete. Generally, this means that the page is visually complete and responsive to the user (user can search, scroll, click links, etc.). However, the browser may still be loading asynchronous content or analytics components. Even worse, some sites deliberately defer loading of prominent content until after "document complete." Another problem is that some types of Web performance tools (called "front-end optimization" packages) can defer execution of key page components until after "document complete." This can mean the page looks complete but, in reality, if a user clicks on links or tries to scroll, the site doesn't respond.
"Visually complete" is the moment that all visual elements are painted on the screen and visible for the user. Note that visual completeness is not the same as functional completeness. I touched on why this is so in the above section. But suffice it to say, this is not a great way to measure performance.
Start Render (or Render Start)
The "start render" event is fired in the browser when something (anything!) is first painted on the screen. The paint event may be the whole page, but it could instead be a single word, single image or single pixel. That may not sound significant. After all, if the content is not there and the user can't interact, then what is the value? Keep in mind that, before "start render" fires, the user is staring at a blank white browser screen or, worse, the prior page from which the user just tried to navigate away from. From the user's perspective, "start render" is the moment that the website is clearly working properly. There is significant evidence that abandonment (bounce rate) is correlated very strongly with slow "start render timings. Arguably, "start render" is the most important metric of all.
When the browser requests the base page, that request must traverse the Internet (whether or not a CDN is in play). The request arrives then at the hosting facility, which must fetch (or assemble) the page. Then the process goes in reverse. The response must traverse the Internet again back to the device requesting the page. "First byte" is the time it takes for the first byte of the response to reach the browser. So, "first byte" is a function of twice network latency plus server latency. Other factors, like packet loss, may also impact this metric. "First byte" is transparent for your users. However, the metric is still important because it is critical path for all browser functions.
The "speed index" is a metric peculiar to WebPageTest, a popular piece of testing software. Loosely speaking, the speed index is the average amount of time for visual components to be painted on the screen. Pages with a faster "start render" and a faster "visually complete" would have a greater percentage of the screen painted at any time—so the area above the curve would be less, and the speed index would be less (lower is better). "Speed index," however, faces the same pitfalls as "visually complete" and "document complete," in that it is not a good way to measure the load times of non-visual components, which might still be critical to the user experience.
Real User Monitoring (RUM)
Synthetic Measurement Tools
RUM tools are excellent for measuring performance, but sometimes we really need synthetic measurements—especially for evaluating performance of pre-production environments (code/stack). Synthetic tools pre-date RUM and are more mature from a software development standpoint.
A few of the big ones include WebPageTest, CatchPoint and Keynote Systems. Synthetic tools can very quickly give you an objective view of site performance. The key is to make sure that you understand the limitations of these tools. Synthetic measurements operating in controlled environments cannot approximate live user experiences—particularly not for highly dynamic applications. That said, synthetics are excellent tools for benchmarking performance, because they provide a more consistent point of comparison. So if you are tuning a website or application, then synthetics can be a great way to nail down the efficacy of your Web performance optimization tweaks.
So now you have an overview of some of the key Web performance terms and their relevance to the new dynamic application landscape and can better understand the geekspeak emanating from your Web performance team.