I am looking for theoretical resources (books, tutorials, etc.) to learn about making sound statistical inferences given (plenty of) multivariate website conversion data.
I'm after the math involved, and cannot find any good non-marketing stuff on the web. The sort of questions I want to answer: how much impact does a single variable (e.g. color of text) have? what is the correlation between variables? what type of distribution is used for modelling (Gaussian, Binomial, etc.)? When using statistics to analyze results – what should be considered as a random variable – the web-page element that gets different variations or the binary conversion-or-no-conversion outcome of an impression?
There's plenty of information about different website optimization testing methods and their benefits\pitfalls, plenty of information about multivariate statistics in general, do you guys know of resources that discuss technical statistics in this specific context of website optimization ?
Thanks for any info!
Best Answer
This Microsoft page has quite a few resources.
I suggest you read at least this paper from the page: "Controlled Experiments on the Web: Survey and Practical Guide." It'll give you some starting point about the metrics to measure and convey, things to consider for online experiments (including designing elements of web pages), and the related statistics.
Enjoy! -Al