Anytime we draw conclusions from statistical inference, other process evidence must support the conclusion. Statistical evidence is only half of the voice of the process. The big picture includes a thorough look at the practical significance of the statistical result.
One area that gives many process improvement teams difficulty is the selection of an acceptance level that is consistent with the reality surrounding the process. There are no hard and fast rules that can help to ensure the selection of the best acceptance criteria. This requires the observation of the process, an evaluation of the business’ objectives, an understanding of the business’ economic realities, and most importantly, the CTQs of the business’ customer base. For example, the acceptance criteria for the safety of an airplane might be set at 0.2 instead of 0.5.
Another problem area is in the interpretation of the statistic result. Since the data creates a picture of the process’ behavior, is this picture consistent with reality? Some important questions to answer are:
Does the statistical result make sense within the process’ current reality?
Does the statistical result point the way to defect reduction?
Does the statistical result point toward a reduced COPQ?
Are there any negative impacts associated with accepting the statistical result?
Does the customer care?
A good data detective will always question statistical conclusions. Performing reality checks throughout the statistical analysis process will help to prevent costly mistakes, improve buy-in, and help to sell the recommendations made by the team.
Practical Significance versus Statistical Significance
When hypothesis-testing tools are used, we are working with statistical significance. Statistical significance is based upon the quality and amount of the data. Process significance involves whether the observed statistical difference is meaningful to the process.
This can work two ways. First, a statistically significant difference can indicate that a problem exists, while at the same time, the actual measured difference may have little or no practical significance. For example, when comparing two methods of completing a task, a statistically significant difference is found in the time required to complete the task. From a practical standpoint, though, the cycle time difference had no impact on the customer. Either the team measured something unimportant to the customer, or a larger difference is needed to affect the customer.
The opposite is also true. The team can find that the observed time difference from above is not statistically significant, but that there is a practical difference in customer or financial impact. The team may need to adjust the acceptance criteria, collect more data (i.e., increase the sample size), or move forward with process changes.
When statistical and practical significance do not agree, it indicates that an analysis problem exists. This may involve sample size, voice of the customer, measurement system problems, or other factors.
Pingback: Kylie Batt
Great information! I’ve been looking for something like this for a while now. Thanks!
Great idea for a blog post. It really helped me out. I’m sure it helped a lot of other people as well.
Very nice, gives me a lot of inspiration being right in the midst of a redesign. It’s all in the details and comment forms are definitely overlooked more often than not. Time well spent on this post.
Thanks for the great ideas in this post! I’m glad to see that I’m doing several things right. I hadn’t thought about putting an interactive question in a post – nice tip. I’m pretty new at blogging and your site helped encourage me to get into some group writing contests, projects and carnivals – which I have thoroughly enjoyed! I can always count on your blog for some great ideas for taking things up a notch. It will be fun to think of some ideas for non-family related universal post topics. Thanks for such an informative post!
Great post from an expert and it will be a great knowledge to us and thank you very much for sharing this valuable information with us.
Exceptional webpage. My class mates and I were just discussing this the other night. Also your webpage looks great on my old laptop. Now thats uncommon. Nice work.
Heya, already been reading your blogging site for a long time. I manage a comparable blog page but I keep receiving a pile of spam comments, how can you maintain your blog page so clean?
Awesome examples, gonna bookmark this for later use when I will create my own comments form
Found this very helpful, many thanks for posting this.
I was actually looking for this resource a few weeks back. Thanks for sharing with us your wisdom.This will absolutely going to help me in my projects .
Hands down, Apple’s app store wins by a mile. It’s a huge selection of all sorts of apps vs a rather sad selection of a handful for Zune. Microsoft has plans, especially in the realm of games, but I’m not sure I’d want to bet on the future if this aspect is important to you. The iPod is a much better choice in that case.
No BS and written well, thanks much for the post
Considerably, the post is really the greatest on that worthy topic. I agree with your conclusions and also will eagerly look forward to your future updates. Simply saying thanks will not just be sufficient, for the fantasti c clarity in your writing. I will best away grab your rss feed to stay abreast of any kind of updates. Genuine work and also much success in your business dealings!
Go to welovemountains for a great selection of road bikes.
Hi there! Do you know if they make any plugins to protect against hackers? I’m kinda paranoid about losing everything I’ve worked hard on. Any recommendations?
I’m typically to running a blog and i really admire your content. The article has actually peaks my interest. I’m going to bookmark your web site and keep checking for new information.
This specific article, “Practical Significance versus Statistical Significance
| Lean Six Sigma Process Improvement” indicates that you really understand just what you’re communicating about! I personally absolutely agree with your blog. Thanks a lot ,Genie