Menu
Effective Base
Effective Base
Weighted tables running A/B comparison often don’t show differences as significant, while running the same numbers via an online significance calculator will show the same differences as significant. This document explains the reason for this difference.
Effective Base
Weighted tables running A/B comparison often don’t show differences as significant, while running the same numbers via an online significance calculator will show the same differences as significant. This document explains the reason for this difference.
Effective Base
Background
Weighted tables running A/B comparison often don’t show differences as significant, while running the same numbers via an online significance calculator will show the same differences as significant. This document explains the reason for this difference.
Effective Base (eBase)
Weighted tables can be tested for significant differences using two methods, weighted base method and effective base method. Effective base method is a more conservative method of running significance testing between groups. This technique ensures adjustments made by weights do not produce false positives, helps in reducing the likelihood of significant differences because of weight adjustments, not due to actual data differences.
This technique helps in ensuring extreme weights don’t influence the results disproportionately while capturing the intent of weighting.
Use
Effective base is used only for the purpose of significance testing. Effective base is used as a safeguard against making statistical conclusions from a sample that has been drastically adjusted (using weights) to match target values.
Calculation
eBase = Square (Sum of Weight Factors) / Sum (Squared Weight Factors)
Total responders – 37
Square(Sum of Weight Factors) – 2,385.601
Sum(Squared Weight Factors) – 95.469
eBase = 25
In above example, using weighted base, Group B appears to be significantly higher than Group A at 80% confidence interval. However, when tested using effective base the differences are not significant.
In data tables, use of effective base is considered more effective in deciphering results, helps in ensuring the findings can stand the scrutiny of weighting process.
Effective Base
Background
Weighted tables running A/B comparison often don’t show differences as significant, while running the same numbers via an online significance calculator will show the same differences as significant. This document explains the reason for this difference.
Effective Base (eBase)
Weighted tables can be tested for significant differences using two methods, weighted base method and effective base method. Effective base method is a more conservative method of running significance testing between groups. This technique ensures adjustments made by weights do not produce false positives, helps in reducing the likelihood of significant differences because of weight adjustments, not due to actual data differences.
This technique helps in ensuring extreme weights don’t influence the results disproportionately while capturing the intent of weighting.
Use
Effective base is used only for the purpose of significance testing. Effective base is used as a safeguard against making statistical conclusions from a sample that has been drastically adjusted (using weights) to match target values.
Calculation
eBase = Square (Sum of Weight Factors) / Sum (Squared Weight Factors)
Total responders – 37
Square(Sum of Weight Factors) – 2,385.601
Sum(Squared Weight Factors) – 95.469
eBase = 25
In above example, using weighted base, Group B appears to be significantly higher than Group A at 80% confidence interval. However, when tested using effective base the differences are not significant.
In data tables, use of effective base is considered more effective in deciphering results, helps in ensuring the findings can stand the scrutiny of weighting process.
Contact
Toronto – Corporate Headquarters
100 Sheppard Ave East. Suite 503
Toronto, Ontario M2N 6N5
+1 (416) 549-8000
+1 (888) 224-6198
info@b3intelligence.com
Copyright © 2023 b3Intelligence, All Rights Reserved | Disclaimer | Privacy Policy