The old way of doing software testing was to build something – and then push several versions of it through an internal quality assurance team. This cost big companies a lot of money and time – but it was necessary. The companies who followed this simple step properly gained a reputation among software enthusiasts because their software rarely had any bugs.
This was before the internet, though – now things have changed, and the pace of the world is a lot faster. People demand new features at an incredible rate, and companies have started seeing QA testing as a chore they would rather not go through.
Microsoft has been very busy converting Windows into something that updates pretty much like your web browser – since the launch of Windows 10, we have already had two major upgrades. The latest of these updates was the Anniversary Update – launched exactly 1 year after the Windows 10 release.
Much like a lot of the software releasing today, Microsoft thought it would be a good idea to replace their QA pipeline with a massive crowd of actual users – this is what the Windows Insider program’s purpose is.
This method is not as bad as it sounds, as it also allows a larger amount of hardware configurations to be tested. Microsoft can’t possibly own and access every type of hardware there is, but the consumers are the actual people who run these configurations. So if these configurations are tested during the Insiders phase, it could result in a better product at the end.
However, it did not go as planned – the Anniversary Update released with tonnes of issues, despite being one of the most heavily tested upgrades Microsoft has ever produced. This is a problem Microsoft has to deal with for future upgrades, but for now, it seems the solution is to slow down the rollout.
The staged rollout
Conducting a ‘staged rollout’ is not an uncommon idea when dealing with QA testing like this. They allow you to only expose a certain subset of your users to the new version – and then wait for issues to pop up, so you can plug them. Once done, repeat. This helps reduce the amount of incidents that occur when the software fails.
A lot of people might have noticed that they have not yet officially received the Anniversary Update on their PC’s. There is a good reason for that – that PC’s configuration has not yet been tested by Microsoft via their internal testing, or the Insiders, or the public release.
If users feel adventurous they can obviously install the Anniversary Update manually – this will give Microsoft more data to work with and might help with the rollout for others with similar configurations.
Microsoft expects to finish the Anniversary Update rollout process in three months – but the process started on 2nd of August, so this gives us another month and a half for the full rollout. In another month and a half, all Windows 10 users should be on the Anniversary Update.
This strategy of using real world users for quality assurance testing has many benefits and downfalls of its own – but at the end, this method is supposed to lead to a better experience for most users.