Show notes

Mass democratisation and access to AB test tooling has meant that hundreds of millions of experiments have been run on people, via digital products and services.  Although we are not running pharmaceutical trials and tests, these experiments still raise numerous ethical issues that are shared with research or experiments in a medical scenarios. 

Craig Sullivan joins us to explore the topic. Craig has been experimenting and optimising websites and digital services for decades and has been regular guest on UXPodcast since the start.

We discuss what, if anything, has changed in the decade since Craig first joined us on UXPodcast. We talk about the explosion of mass experimentation and the impacts of the changes we make, the tests we run, and the problem of not testing at all. We talk about the problems of North-star metrics, and the need for ethical training and the processes organisations (and individuals) need to adopt to reduce the ethical gap.

(Listening time: 39 minutes, transcript)

Episode #280: Ethical experimentation with @OptimiseOrDie – Mass democratisation and access to AB test tooling has meant that hundreds of millions of experiments have been run on people – is it ethical? https://t.co/Nk4WIRl410#ux #uxpodcast #podcast pic.twitter.com/oqIVeMCmZ4

— UX Podcast (@uxpodcast) January 14, 2022

References:

• Full transcript for episode 280 
• Craig Sullivan
  • Twitter: https://twitter.com/optimiseordie
  • LinkedIn: https://www.linkedin.com/in/craigsullivan/
  • Medium: https://optimiseordie.medium.com/
• Children mining for cobalt in DRC
• The slavery supported by that device in your pocket (By Per Axbom)
• Perverse incentives (Craig’s comedic LinkedIn post)
• LinkedIn article about A/B tests
• Perverse incentives and the Cobra Effect (Wikipedia)
• IRB (Institutional Review Board)
• Reading tips from Craig:
  • Embrace Failure, Inform Research
  • 3 things you can do before launching your next experiment
  • Ethics, UX and our role in all this
  • We are all being manipulated by testing all the time
  • Who’s the Guinea Pig
  • The Ethics of Online Research
  • The Paradox of AB testing
  • Driving better ethics and consumer outcomes in tech
  • On the Ethics of AB Testing
• All 7 UX Podcast episodes featuring Craig Sullivan
• Backstage mailing list (show notes to your inbox, plus extra stories, news and offers: Sign up here
• Enjoying UX Podcast? Contribute with your time as a volunteer, or with real cash to help keep the podcast running: Support UX Podcast