WooCommerce Payments is an open-source plugin designed exclusively for WooCommerce stores. On the quest of giving our merchants the best possible experience, we write a lot of E2E tests to cover critical flows – this gives us confidence against regressions and allow us to iterate faster.
However, sometime last year we realised that due to the nature of E2E tests, our CI pipeline sometime ran for over 40 minutes! Since a green from CI was required for merging PRs, this ended up slowing us down instead.
Thus began the quest to speed up the CI pipeline – this talk will walk you through the different things we tried, how we reached those conclusions and how it ended up affecting the overall pipeline in the end.