eBusinessBlog.org Leveraging marketing & technology to solve business problems.

Six Sigma Web Development and the steps 2-5 of DMAIC – Measure, Analyze, Improve, and Control

Yesterday I covered step 1 of DMAIC, the "Design" phase as a part of the larger Six Sigma for Web Development topic. Moving right along to step 2 after design phase is implemented is the task of measuring the success or usability of your design. Since the design phase of a website usually isn't released to the public, you must execute qualitative and quantitative surveys/studies.

What is a qualitative survey?
A qualitative survey or study is where you have a small group of participants who you have carry out tasks on your website in a one-on-one setting. This is most often a usability study ran by a facilitator (or yourself). A group of 10 people over the course of two days will give you plenty of data to determine whether or not the design phase of DMAIC was a success or needs considerable work. The point of these surveys or studies is to do a deep dive with each individual spending 60-90 minutes with them understanding how they navigate the site, identifying their frustrations, and determining whether or not they can even complete the task at hand.

You will pick up a lot of small "fixes" from a qualitative study that will add up to an excellent list of incremental improvements you can make. You should also identify any major disconnects the site has in its navigations or the tasks that individuals are asked to complete.

What is a quantitative survey?
A quantitative survey is more about volume, or quantity of responses. These are going to be questions like, "On a scale of 1 to 5, 5 being very satisfied, please rank your overall satisfaction with this website." You won't get the deep dive like in a qualitative study, but you will get a very good overall ranking from a larger group of participants. Usually, you want about 200 responses to be statistically significant in a quantitative survey.

Perform a qualitative study first.
In my experience, I would recommend performing a qualitative study first. If you do a quantitative study first you may see your overall rankings suffer, but you won't really know why. It essentially reinforces the point that you will need to do a usability/qualitative study anyway. The qualitative study is always going to deliver rich feedback that your team can prioritize and execute. This data is so powerful that once it is implemented, the quantitative study should help identify any remaining problem areas. If there are significant drop-off areas or low rankings, you can individually tackle and evaluate these sections.

Structuring the Measure, Analyze, and Improvement phases of DMAIC
Additionally, the use of web analytics to see how these people from the quantitative study are navigating your site is an added bonus. This is how I would structure the these three phases of DMAIC:

1. Ensure your conversion funnels being tested are fully functional and compatible on the platform in which you will be testing.
2. Execute a usability (qualitative) study
3. Evaluate results of usability study, prioritize development and design tasks, and implement changes
4. Integrate web analytics into your site at this time -- the reason why I say to do this here is because your navigation may significantly change as a result of the usability study and any custom integration points you worked on before may need to be redone at this phase. Hopefully, you've got the navigation dialed in at this point and won't be changing too much more than some interface design components at this point.
5. Execute a quantitative study of 200+ participants. If you work for a large company, soliciting employees from other divisions who are not close to your products is an ideal way to do this. Otherwise, a friends and family test will also suffice. The key is getting people to perform tasks on the website who are not familiar with your products. If you have your own employees do the testing, the results will be skewed because they will not be relying on the website for product information -- generally, they already know enough about your products to bypass any pain points the website may have.
6. Evaluate results of quantitative study, prioritize tasks, and implement any necessary changes. You may be able to launch your site at this point while continuing to implement rolling changes based on the quantitative results.

Ongoing quantitative studies (The "Control" phase)
Consider companies like OpinionLabs to implement an ongoing quantitative study for your site. With their software, you can even obtain qualitative results from people from individual pages or sections of your site. This information is powerful and the ongoing measurement of satisfaction of your site will be an important way to continue to improve overall satisfaction and conversions.

Bricks and mortar retail stores do not just put products on the shelf and leave them for eternity. They are constantly trying new end-cap displays, custom product displays, promotions, and adding/removing products to optimize their layouts. Website visitor's usability experience and requirements will also change over time. The same sites built in the late 90's are laughable today. You need to be able to change with the times and this is why the control phase is extremely critical.

Six Sigma Web Development and the first step of DMAIC – Define

As a continuation on the topic of applying Six Sigma methodologies towards web development, design, and usability, this post focuses on step 1 of the DMAIC process, "D" (Define):

In the "Define" phase of web development, this is where you identify the key components your customers need in order to navigate a conversion funnel. This concept can be applied to many types of websites, but here is an example of how this applies to an e-commerce website:

Defining attributes for e-commerce sites
A primary goal of an e-commerce site is to generate revenue. This means you'll need a storefront, the ability to add products to a cart, and the ability to securely purchase these products online. Compatibility and usability issues aside, this is usually a significant area of oversight for many web developers, marketers, and managers.

How would your online store compare to bricks and mortar?
If you think of a bricks and mortar business, you can associate their aisles of products with your online store. Moving further down their path, you will also find shopping carts to hold products and cash registers to process transactions. What many sites fail to realize is the sales component of a store. A bricks and mortar business has the benefit of sales associates who can help customers with questions about products and with how to find products in their store.

Websites lack the human component. Don't just focus on the tail end of your conversion funnel.
Websites on the other hand lack this human interaction, so it's up to the "Define" phase of Six Sigma's DMAIC process to identify the key components that will help drive consumers to the sales funnel. This is typically going to be a product search engine, product comparison tool, and a product catalog (outlining more in-depth information than the online store). If a customer can't figure out your product or service if it were sitting on a shelf in a bricks and mortar store, then don't expect them to be able to understand it online.

Leverage retail environments when building an online experience
If you have an advantage of selling products online that are also sold in retail stores, you can piggyback off of your learnings from the retail experience. What benefit does a sales associate play and what is the process that the associate walks the customer through the conversion funnel? At which point does the associate hand the customer off to the purchasing portion of the funnel? Use this experience and information and convey it online -- it's a necessary feature.

Failure to define the complete components of a conversion funnel will be identified with qualitative surveys
Avoid tunnel vision from the beginning when building a site. Whether it be e-commerce, social media, etc., don't forget that the majority of your visitors will not be experts in the product or service you are trying to sell them or get them to use via your website.

My next post will discuss quantitative and qualitative feedback for the "Measure" phase of the DMAIC process.