SMALL TALK

What’s the fastest way to understand your user? Talk to them. To get the team to
empathize with Carvana’s users I had the design team listen to our inbound customer
service calls. To my surprise about 80% of callers also agreed to be interviewed after
speaking to a customer service advocate.

We learned a lot speaking to a wide demographic. Our biggest takeaway–
storytelling. Users lacked a narrative that simple data didn’t provide. Some users
were subject matter experts while others needed explanation of some basic
automotive terms. Another common thread: ‘help reassure me that I’m making the
right choice’. We heard this repeatedly.

Through customer interviews, we learned that most of our users had some level of
familiarity with the car they were looking to purchase. Customers collect vehicle
knowledge through friends and family or researching the manufacturer’s website.

The team also interviewed key stakeholders within Carvana. A site redesign is a
massive undertaking that evokes a variety of feelings throughout an organization.
These internal conversations with cross functional teams allowed the design team to
get a better grip on the organization’s collective hopes and reservations. The
feedback was a mix of proposed architectural changes, marketing shifts and
messaging opportunities. Here’s an aggregate of what stakeholders were hoping for.

Above: Carvana’s 360º experience lacked both hierarchy and clear calls to action. Iconography was accompanied by rollovers and
margin instructions.

Above left: Even team members had trouble distinguishing the difference between section headers and tappable vehicle images.

Above right: Carvana’s finance calculator presented a fun metaphor but also challenged users with unexpected controls and a difficult
way input specific numbers.

Each team member presented their site audit. Afterwards, the group collectively
synthesized pain points into themes– areas of biggest need. This gave the team
tactical usability issues to design against but also bigger picture guiderails to
reference throughout the redesign effort.

A PEEK AT THE NUMBERS

Analytics were at the core of helping the design team make key decisions. We
needed to know where traffic was coming from and how customers were interacting
with the site. Metrics were pulled for all the core pages. Through a partnership with
Carvana’s Analytics team we were better able to understand macro conversions, in-
page analytics and navigation summaries. Key features and marketing modules were
particularly scrutinized for engagement.

Above: Understanding macro analytics gave the team a knowledge base of user’s devices– where customers were coming from and
duration on key pages.

Above: Digging into flow analytics helped the team understand potential blockers. Here we saw little engagement with our Vehicle
Detail Page subnavigation on mobile. It also became clear that we needed to consider a different hierarchy with our highest engaged
module on the lower half of the screen.

Additionally, working with Product and Analytics the design team pulled together a
comprehensive list of Key Performance Indicators ahead of our redesign. This gave the
UX team context for how the business would be measuring success for the redesign.

EYE ON THE COMPETITION

The team utilized usertesting.com to have users take a series of task based tests on
Carvana and three competitors. Participants were given the same tasks and asked
the same questions across the three sites. They were asked to use the ‘Think out loud’
method as they completed tasks. The net result gave the team some insights into key
differences between our experience and that of our competitors.

Above: Improving our photography was one of the many signals the team received during competitive testing.

The team also analyzed different competitors with comparison charts. This gave the
team and stakeholders a series of easy to scan differences between Carvana and the
competition. This helped make recommendations to market our products better and
assess where we could create product opportunities.

STAKEHOLDER WORKING SESSIONS

Over the course of two days, I led a cross functional group of stakeholders in a
series of generative brainstorming sessions. Team members were given a variety of
exercises. This included warm up exercises to get them prepared to think like users,
crazy eights and prompts in feature prioritization.

Stakeholders were given homework at the end of the first session. They were asked
to come back and share best in class experiences as well as provide examples of
subpar experiences on our site.

Above: A stakeholder’s example of best in class error messaging vs. Carvana.

Above : Examples of effective infographics paired with a 3 dimensional product.

ALL WIRED UP

After 4 weeks, the Learning phase was completed. Myself and the team began to
sketch and create wireframes. After a few days we brought product stakeholders in
to validate our ideas and whiteboard alongside.

Wireframes and sketches gave us the opportunity to establish changes to the site's
navigation, work through new features and hierarchies, as well as display intended
functionality.

Above: Wireframes showing and idea for an educational section on Carvana’s homepage.

UPDATED STYLE GUIDE

Once we had directional agreement on navigation, content and functionality, the
design team began to sketch out a design language. The updated style guide
modernized many of the existing patterns already on the site. The team took the
opportunity to modernize, consolidate and take accessibility into account.

PROTOTYPE TESTING

As soon as initial layouts of core pages began to take shape, myself and the UX team
began to create clickable, high fidelity prototypes. We utilized InVision and
usertesting.com to test key flows. Users were given the same key tasks we tested in
the Learning phase. Searching, saving favorites, sharing and locating key vehicle
information were among the tasks we gave users.

Additionally, more complex prototypes were created using the prototyping tool
Principle. The team utilized these prototypes to replicate more complex functionality
and test newer, more experimental features. In order to get detailed feedback, we
conducted in person usability testing with our Principle prototypes.

Above: A Carvana customer scrolls through proposed categories using a prototype of Carvana’s homepage.

The insights we gained from both types of tests helped the team refine key
functionalities and validate assumptions. We learned that most participants wanted
to spend less time learning about Carvana and more time searching for vehicles.
Participants reacted really well to our newer visuals and layered information. When
asked about key value propositions after the test most participants remembered a
few but almost nobody remembered them all. This was an eye opening moment that
allowed us to pivot and add contextual value props throughout the search and
vehicle detail pages.

LAUNCH OF KEY PAGES

With Home, Search and Vehicle Details being the first pages redesigned and tested,
the organization decided to develop and launch these first. This allowed us to
test the core experience in the wild and make needed changes prior to redesigning
all ancillary pages and launching everything at once.

Above: Version 2.0 of Search Results.

Above: Version 2.0 of Vehicle Details on the Vehicle Details Page.

SUCCESS METRICS

Over the course of several months, the design team tracked analytics and made a
few layout and functionality changes. Overall, the launch of our core pages was quite
successful and increased our main KPIs.

KEY LEARNINGS

Although the redesign was successful, myself and team had some key takeaways. The
design team conducted a post mortem. Below are some insight that came out of it.