New Applause Survey Shows How Bias Can Infiltrate AI

New Applause Survey Shows How Bias Can Infiltrate AI

A few weeks ago, the Applause team attended the North American AI & Big Data Expo in California. Having just announced our new AI training and testing solution, the event was the perfect time to illustrate how easily bias can permeate an AI algorithm, and how a large and diverse data set is the pathway to avoid this problem.

Our hypothesis was that a global – and more diverse – group of individuals would produce data that was less biased. To test this theory, we created a simple one-question survey: What is your favorite brand of coffee?

Two different data sources

We first surveyed our global community of vetted real-world testers, then asked the question to attendees of the AI & Big Data Expo to see how the results compared. We expected that responses at the tradeshow would be more homogeneous due to event-goers coming primarily from the West Coast and working in technology roles. Our community, on the other hand, is universal and features people from all backgrounds, genders, races, work roles and locations. We hypothesized more diverse answers from them.

Though we expected more varied answers from the community, we were still surprised by the difference in responses between the community and the audience at the AI & Big Data Expo.

The West Coast bias was clear from the get-go in the show floor results.

West Coast bias in effect

The West Coast bias was immediately clear from the show floor results. Starbucks was far and away the favorite, accounting for 37% of total responses, followed by California-based chains Peet’s and Blue Bottle. Those three West Coast brands accounted for 65.5% of total responses, while only 17 brands received at least one response.

Comparatively, 87 different brands received at least one vote in the community survey. Starbucks was still a top choice, but it ranked second to Nestle and was followed closely by Lavazza – two companies headquartered outside of the United States. The top 10 companies in the community survey accounted for only 62.5% of responses.

Survey results: What is your favorite brand of coffee?

favorite coffee survey results: Starbucks 39.1%, Peet's 15.5%, Blue Bottle 11.9%

AI & Big Data Expo

favorite coffee survey results: Nestle 17.4%, Starbucks 15.4%, Lavazza 10.6%, even break down of the 7 remaining

Applause Community

The diversity of answers is immediately noticeable when you compare these two images, as the community provided a longer list of brands and more global brands. These results more accurately represent the coffee drinking habits of a worldwide customer base.

I broke down these results even further, and explained Applause’s own journey with AI, in this webinar linked below. Make sure to tune in to see the full results and learn why the human element is still essential for AI success.

Whitepapers

Your Keys to Combating Bias in AI

Whether intended or not, bias is built into every AI engine. See where you need to be vigilant and the steps you can take to eliminate bias from your AI initiatives.

READ NOW
Want to see more like this?
Dan Cagen
Dan Cagen
Product Marketing Manager
Reading time: 5 min

Digital Quality Matters More Than Ever: Do Your Experiences Keep Customers Coming Back?

Take a deep dive into common flaws in digital experiences and learn how to overcome them to set your business apart.

4 Ways to Get Maximum Value from Exploratory Testing

Well-planned exploratory testing can uncover critical issues and help dramatically improve the customer experience. See how to guide testers to where exploration can yield the greatest returns.

3 Keys to an Effective QA Organization

Get your internal, external and crowdsourced testers on the same page

What is the Metaverse? And What Isn’t It?

It’s not far-flung sci-fi anymore — the metaverse is here, and it requires companies to rethink their approach to UX and testing

Why Machine Learning Projects Fail

Read this article to learn the 5 key reasons why machine learning projects fail and how businesses can build successful AI experiences.

How Localization Supports New-Market Launches

Success or failure in a new market is all about how you resonate with customers — don’t skimp on prep work