@Data4DecisionMaking
What alpha level do you recommend?
@PhilipThonbo
I have 2 tests i ran on the same "change" one AB test with a broad goal criteria (ex all devices) and another AB test with anarrow criteria (ex. tablet traffic) - the broad had 10.000 conversions and a slightly lower CVR. (70% significance) The narrow had a bit higher CVR. (95% significance) but also much fewer samples. What would you then conclude is winning? 1000 (95% signif.) vs 10.000 (70% signif.) ? It seems like the formula is not bulletproof when very low sample sizes can get very high significance just because they show more extreme results.
@DillipRout-hm4jk
Could you please explain multivariate statistical A/B test?
@Guilopes99
Very nice and clear explanation on how to use the tool. However, they have a sample size calculator that for the same experiment you're doing, says you need way bigger samples... What should we do?
@GaganKPolska
A/B test idea: Just change the word in the title from 'dummies' to 'beginners.
@MarvinTokhai