A/B and Multivariate Testing

I have further argued that even without access to personal data, massive amounts of data cause an ethical problem related to manipulation. This manipulation is related to the ethical implications of A/B and multivariate e-commerce optimization testing (Sylvia IV, 2010). These techniques, which allow e-commerce websites to test different versions of a page to improve outcomes like sales or reduce abandoned shopping carts, might seem innocuous. However, I believe there’s more to consider.

I’ve been involved in owning or managing e-commerce websites for 20 years, and I first became aware of the issues surrounding this sort of testing in the first decade of my career. It was around this time that Google’s free Web Analytics software launched and was available for free to the general public. This allowed virtually any site that wanted it to run these A/B and multivariate tests and collect data on them. I saw the power of these tools first hand as I integrated them into the site I was managing. It was witnessing this power that first raised my ethical concerns. This is part of what brought me back to school to pursue my master’s degree, and later my doctorate.

I examined these practices through various ethical lenses and I’ve found that they can lead to manipulative site design. The goal is to subtly encourage consumers to spend more. Although another viewpoint might see these practices as aiding consumers—making websites more user-friendly or easier to navigate—I think the reality is more complex. The goal, much like the field of advertising in general, is to create new desires to purchase products you don’t actually need. But this iterative process lets websites get really good, really quickly at persuading you in ways that are not at all transparent. How could you possibly imagine that the color of the checkout button on your favorite website makes you more likely to actually complete a purchase unless you’ve studied web design or communication theory?

The primary issue that I took at the time with these practices was that the why didn’t matter. Why does a certain size and color button make people spend more? Why does a certain shade of blue make people more likely to click a link? This type of testing cannot answer that question. As we transitioned into the age of big data, that problem has only become more pronounced. Big data is very good at making correlations between things, but not able to explain why those correlations exist.

And this brings us face-to-face with the difficult theoretical questions we must all face in the age of big data.