Optimising User Experience: Simyo's Success Story with INP Improvements

Challenge

Like many other sites, Simyo did really well on First Input Delay. However, based on synthetic data, it was thought that responsiveness could be better, especially for mobile visitors. Although Simyo only had synthetic data on initial load, it was suspected the high scores for metrics like Total Blocking Time (TBT) and JS Long Tasks were affecting post-load interactions.

Based on collected synthetic data we were able to gain some valuable technical insights. It seemed that the main cause of these INP issues was the use of third-party tooling and tracking. This is probably related to the high level of JavaScript used by third-party vendors. However, we weren't 100% sure about this because we didn't have any data from post-load interactions. Our research, optimisation and removal of third-party resources didn't fully solve the responsiveness problem because it was based on synthetic data for the initial load only.

The introduction of the Interaction to Next Paint (INP) metric as a potential alternative to First Input Delay as a responsiveness metric has proven to be a valuable insight. Initial suspicions were confirmed by the Chrome User Experience Report (CrUX): the responsiveness of page interactions for mobile users was found to be below expectations. With this data, we also knew which pages had the highest INP scores.

The biggest challenge was to work out which specific interactions were delayed and by what causes.

Solution

The CrUX API provided insight into the INP values of the origin, page types and specific URLs. All pages were also monitored using RUM tooling, but INP measurement was not yet available. More specific insight was needed and this was found by using RUMvision, a RUM tool that makes use of the Google Web Vitals library. Not only was RUMvision able to monitor INP per page, the ability to provide attribution data using the Google Web Vitals library allowed Iron/Out to quickly identify which interactions were causing delays for mobile users.

While learning about specific HTML elements and INP breakdowns provides very valuable insight, the introduction of the Long Animation Frames (LoAF) API in Chrome made a significant difference. The Long Animation Frames API is a relatively new tool that helps developers identify and fix performance issues caused by long animations. According to Google, LoAF helps provide a better understanding of slow user interface (UI) updates. It gives insight into the duration of animations and their impact on page responsiveness. This can be a useful tool to identify slow animation frames which may affect the Interaction to Next Paint (INP) metric. It can also be used to identify other UI jank which affects smoothness.

As Product Owner of the Simyo website, I'm impressed with the way Iron Out worked with us on this project. Their ability to clearly identify and define actionable improvements was instrumental in improving the performance of our site. The significant improvements in our INP metrics are a testament to the effectiveness of our collaboration. It reflects a shared commitment to delivering an exceptional user experience.

Lukas van Dijk, Product Owner Simyo

At the end of 2023, LoAF was only available in Chrome as an Origin Trial. However, RUMvision was already using the LoAF API while in Origin Trial, so we could already make use of it. RUMvision offers a third-party dashboard that shows the "pure JS execution time" of first- and third-party scripts. With this data in hand, we were able to pinpoint the source of performance bottlenecks. 

×

Example of the third-party dashboard in RUMvision

It became evident that the responsiveness issues were caused by a global first-party resource. Based on the analysis of performance traces conducted by running performance profiles in Chrome Devtools, it emerged that this resource was pushing data to Google Tag Manager (GTM) when a user was interacting with elements on a page. While it is relevant to analyse how visitors use the site, this is not critical for the visitor during interactions. Visitors expect an immediate response in the form of a visual update as they interact with the elements on the page.

In collaboration with developers at Simyo, we provided and applied solutions to prioritise UI updates within the browser's main thread and only sending data to GTM when the UI has been updated. This ensures that the user experience meets visitor expectations.

Results

Following the implementation of these optimisations, we achieved a significant improvement in the INP for mobile traffic, with more than 75% of users experiencing an INP of less than 200 ms. This resulted in a notable reduction in the INP for the origin, from 413 ms to 150 ms, representing a 64% improvement.

×

Percentiles (Chrome UX Report)

×

Distributions (Chrome UX Report)

The experience for desktop users also saw a notable improvement of 49%, with INP dropping from 102 ms to just 52 ms.

In our communication with clients, we report a UX score for pages and origins. This score is based on the key factors that matter to website visitors. It is easy to understand and provides insight into whether the site is fast and works well for everyone. The UX score enables IT staff and business leaders to make informed decisions that enhance the user experience of their websites. 

This improvement in INP has led to the origin achieving a UX Score of 100 points, reflecting the enhanced user experience for mobile users. A total score of 100 indicates excellence across loading speed, visual stability, and responsiveness.

×

While the numbers speak for themselves and show that the user experience has improved, the videos below show you how the interaction on a ‘handset’ page (product detail page) has improved significantly. In both videos you can see that a user selects different colours for the handset. After tapping on a colour, this choice is confirmed in the colour picker and the image of the device is rendered in the corresponding colour. Before the changes, there was a noticeable lag when updating the UI, but after the optimisations, the experience is so much smoother.

Before optimisations had been implemented

After optimisations have been implemented

Both tests were carried out on the exact same device, a budget Android smartphone (Samsung Galaxy A40) with limited processor power, making the chances of INP issues relatively high. However, we know that many other users visit the site with devices using the same or similarly less powerful processors.

Conclusion

In conclusion, Simyo's dedication to improving the user experience on their website has been evident through their focus on speed and responsiveness. By identifying and addressing issues with delays in page interactions, particularly for mobile users, they were able to significantly enhance the overall user experience. The implementation of solutions led to a remarkable improvement in key metrics such as Interaction to Next Paint (INP), resulting in a UX Score of 100 points for mobile users. With these improvements, Simyo has demonstrated their commitment to providing an excellent digital experience for their customers, further solidifying their reputation as one of the best mobile providers in the Netherlands.