‘The Road to PWA’ Shane Osbourne at Magento Live Europe 2018
I’ve given a few talks on front-end performance already this year at events like Reacticon in the Netherlands & Meet Magento in the UK – so I’ve seen first-hand how positively people are reacting to JH sharing knowledge on this topic.
Nothing could have prepared me, however, for the sight of a room so full that people were resorting to standing around the edges – and in some cases even being refused entry altogether!
As I was squinting through the bright lights during my opening slides at Magento Live Europe, I could see every seat was taken and the production team were beginning to turn people away – it made me realise just how important this topic was and how much people craved information about it.
Whilst JH push forward with PWA development at full speed, on other projects we also recognised a real need to improve the Magento Platform for all clients new and old who would not be going ‘full PWA’ immediately – essentially this means all clients still utilising the existing Magento 2 Frontend.
The day-to-day reality of client work means that rushing to adopt the latest trends in tech for the sake of it, is just not something any responsible agency would do. So whilst we’re going all-in with the PWA Studio from Magento for new projects, it doesn’t mean that we’re turning our backs on all those other projects that are either currently live, or going live soon with the regular Magento 2 Frontend.
It’s taken a huge amount of time and effort to arrive at the solutions I shared in my talk in Barcelona – almost a year of experimenting with multiple techniques; some radical and some simple.
There are other micro optimisations to be made, as always, but this is by far the technique that will give the biggest improvement – reducing TTI (Time to Interactive) on mobile by 15 seconds on an average product page being one of the highlights.
— Matthias Glitzner-Zeis (@mzeis) October 10, 2018
Discovering the real problem
It turns out that the RequireJS Optimiser is a mature, battle-tested, robust tool that works perfectly well. So what makes it so difficult to use within Magento? There must be something about this tool that makes it so incredibly hard that almost no-one in the Magento community is actually using it in production…
Well, we found through our research that it’s really only down to a single issue: actually producing the configuration for the optimiser.
There are no issues running the tool itself, that works extremely well – but due to the dynamic nature of Magento and the way it uses modules on the frontend, we need to give the optimiser some intricate knowledge about things, such as which modules should be included in each bundle, which modules are ‘critical’, and which modules should make up ‘additional’ bundles that can be loaded on demand.
Just take a look at how much code is needed here – that file is a reference implementation of the config needed to optimise the default Luma theme – it’s over 452 lines of hand-crafted JSON configuration ?
It actually works remarkably well – a site optimised with this technique is going to be just about as fast as you’re ever going to get a Magento 2 store to be.
Along with taking too much time & effort, the probability of errors creeping in is drastically increased since human-error will inevitably affect anything that’s ‘handcrafted’ like this.
— Ian Cassidy (@iancassidyweb) October 10, 2018
Towards a Solution
Getting bundling to work, in the most optimised way possible has been a much larger challenge than we anticipated. Along this journey we’ve explored multiple avenues – we looked at static analysis, partial static analysis, using Webpack, full automation in the browser and a number of others. The technique I presented in Barcelona however, is the one we’ve had the most success with to date.
At a high level, the concept is quite simple – we’ve already seen that producing the configuration is the hardest part of this process, and the reason that’s the hardest part is that for any page in a Magento website there may be different modules being used. So we thought, what if you could accurately track every module that’s used on each individual page?
If you could do that, then you could look at the list of modules used across many pages, and from it determine the modules that are ‘shared’ – these would become your ‘critical’ modules that should be loaded on every page.
With this tracking information, you could determine that something like the Photo Gallery is NOT used anywhere but the product page, and therefore you can exclude it from the main critical bundle, putting it in a product-page-specific bundle instead.
This process then would repeat for any pages you should choose to have their own specific bundles – these tend to be Homepage/Product Page/Basket/Checkout, but the list could be as large as you wanted.
The point being, that by using real-world data on the usage of modules across pages, you’d be able to solve the hardest problem Magento 2 developers face when trying to optimise their sites – generating that damn configuration for the optimiser!
— John Knowles (@knowj) October 10, 2018
Proof of Concept
The ‘solution’ presented above is more of an idea than anything else (that’s why this post is not a detailed look at how to actually implement it). It’s just an encoding of the realisation that followed many months of banging our heads against the wall trying to figure out how this final pain-point could be solved.
The tools, framework, and libraries used by Magento are not the problem – it’s just the generation of the configuration, and that can be solved if you can keep a track of every module used across multiple pages.
We do have a POC project that attempts to highlight this idea – it’s what we’ve been using in our own projects. Feel free to ask questions at that repo if you’re unsure about anything regarding the approach – we’re eager to iterate on this and make it more useful.
There’s still a small amount of friction, even with the tools that we’re open sourcing – this is mainly around the fact that you still need to exercise the frontend application to get the bundling correct. But after being inspired by some of the Q&A questions I received at the end of my talk in Barcelona, we think there’s a baseline optimisation that could be applied to all Magento 2 stores that have the potential to improve performance by approx 50% with little to no work required.
@rowan_m gave me this idea – is there a ‘baseline’ optimization we can apply to all @magento 2 sites (approx 50% faster), from nothing more than reading requirejs-config.js… then it could be a website, and it could be built in @rustwasm and share code from the CLI tool… pic.twitter.com/w04ssGZEGI
— Shane Osbourne (@shaneOsbourne) October 28, 2018