Six Sigma Web Development and the steps 2-5 of DMAIC – Measure, Analyze, Improve, and Control
Yesterday I covered step 1 of DMAIC, the “Design” phase as a part of the larger Six Sigma for Web Development topic. Moving right along to step 2 after design phase is implemented is the task of measuring the success or usability of your design. Since the design phase of a website usually isn’t released to the public, you must execute qualitative and quantitative surveys/studies.
What is a qualitative survey?
A qualitative survey or study is where you have a small group of participants who you have carry out tasks on your website in a one-on-one setting. This is most often a usability study ran by a facilitator (or yourself). A group of 10 people over the course of two days will give you plenty of data to determine whether or not the design phase of DMAIC was a success or needs considerable work. The point of these surveys or studies is to do a deep dive with each individual spending 60-90 minutes with them understanding how they navigate the site, identifying their frustrations, and determining whether or not they can even complete the task at hand.
You will pick up a lot of small “fixes” from a qualitative study that will add up to an excellent list of incremental improvements you can make. You should also identify any major disconnects the site has in its navigations or the tasks that individuals are asked to complete.
What is a quantitative survey?
A quantitative survey is more about volume, or quantity of responses. These are going to be questions like, “On a scale of 1 to 5, 5 being very satisfied, please rank your overall satisfaction with this website.” You won’t get the deep dive like in a qualitative study, but you will get a very good overall ranking from a larger group of participants. Usually, you want about 200 responses to be statistically significant in a quantitative survey.
Perform a qualitative study first.
In my experience, I would recommend performing a qualitative study first. If you do a quantitative study first you may see your overall rankings suffer, but you won’t really know why. It essentially reinforces the point that you will need to do a usability/qualitative study anyway. The qualitative study is always going to deliver rich feedback that your team can prioritize and execute. This data is so powerful that once it is implemented, the quantitative study should help identify any remaining problem areas. If there are significant drop-off areas or low rankings, you can individually tackle and evaluate these sections.
Structuring the Measure, Analyze, and Improvement phases of DMAIC
Additionally, the use of web analytics to see how these people from the quantitative study are navigating your site is an added bonus. This is how I would structure the these three phases of DMAIC:
- Ensure your conversion funnels being tested are fully functional and compatible on the platform in which you will be testing.
- Execute a usability (qualitative) study
- Evaluate results of usability study, prioritize development and design tasks, and implement changes
- Integrate web analytics into your site at this time — the reason why I say to do this here is because your navigation may significantly change as a result of the usability study and any custom integration points you worked on before may need to be redone at this phase. Hopefully, you’ve got the navigation dialed in at this point and won’t be changing too much more than some interface design components at this point.
- Execute a quantitative study of 200+ participants. If you work for a large company, soliciting employees from other divisions who are not close to your products is an ideal way to do this. Otherwise, a friends and family test will also suffice. The key is getting people to perform tasks on the website who are not familiar with your products. If you have your own employees do the testing, the results will be skewed because they will not be relying on the website for product information — generally, they already know enough about your products to bypass any pain points the website may have.
- Evaluate results of quantitative study, prioritize tasks, and implement any necessary changes. You may be able to launch your site at this point while continuing to implement rolling changes based on the quantitative results.
Ongoing quantitative studies (The “Control” phase)
Consider companies like OpinionLabs to implement an ongoing quantitative study for your site. With their software, you can even obtain qualitative results from people from individual pages or sections of your site. This information is powerful and the ongoing measurement of satisfaction of your site will be an important way to continue to improve overall satisfaction and conversions.
Bricks and mortar retail stores do not just put products on the shelf and leave them for eternity. They are constantly trying new end-cap displays, custom product displays, promotions, and adding/removing products to optimize their layouts. Website visitor’s usability experience and requirements will also change over time. The same sites built in the late 90′s are laughable today. You need to be able to change with the times and this is why the control phase is extremely critical.
No comments yet, be the first.