Pelikan
Live pilot experiment to understand customer behaviors related to autonomous vehicle delivery services.
Challenge:
We set out to answer a complex question: Could a delivery service that prioritizes customer terms be a differentiator, without compromising on business viability?
To do this, Ford needed to figure out how autonomous vehicles could create unique value in moving goods for recipients and retailers through differentiated experience.
We created a Pelikan, a fictional start-up developed to explore human-centered parcel delivery in a community in San Francisco.
Outcome:
Our work informed the go-to-market strategy for a number of initiatives in the moving goods space that Ford is currently evaluating. Specifying an approach that leveraged Ford's unique capabilities, we outlined user experience requirements for a larger scale service based on Pelikan and did an analysis on the business to demonstrate a path to profitability. At the end of the project, we presented findings to the CEO of Ford and the top executives of Ford’s AV LLC business team.
Timeline:
October 2018 to December 2019
Role:
Project Lead, Senior Designer
Core Team:
Design Researcher
Data Scientist
Business Designer
Build Team:
Interaction Designer
5x Software Developers
Electrical Engineer
Mechanical Engineer
Pilot Team:
2x Software Developers
5x Contract Operations Staff
Problem Context
Moving goods is big, broken, and current systems prioritize efficiency over customer preferences. There is an opportunity to create value by introducing something new.
E-commerce revenue (US) has surpassed $500B and is poised to grow another 50% by 2023.
Last mile is big business. It can account for up to 40% of total supply chain costs.
As sales steadily shift to online channels, customers are being let down by rigid logistic systems.
Hypothesis
Based on prior work led by D-Ford, the company had a hypothesis that giving customers the ability to choose how, when, where they want to receive their items is the key to creating a compelling experience.
To rigorously test our hypothesis and gather the evidence necessary for Ford to commit to a new offering, we had to capture unfiltered feedback from repeated interactions. We created Pelikan to explore human centered parcel delivery in an urban environment.
Early Testing
Knowing that we wanted to run a learning experiment to better understand this problem, we did a few lower fidelity tests to define elements of the experience, service requirements, and experiment variables.
Service & Operations
We coordinated package delivery for all inbound deliveries at IDEO SF’s office. This gave us the chance to iterate on the service and operational process.
User Experience Via Text
we used an automated text message flow as a proxy for an app. Users selected from delivery options, received instructions on where to find the delivery vehicle, and details on how to access their items.
Delivery Vehicles
We built two delivery vans out of foam core and plywood. These relatively quick prototype vehicles taught us a lot about the physical experience.
A user retrieves her package from one of the Pelikan vans.
To gain access to the curbside van, we asked users to "scan" their order confirmation.
We interviewed all of our enrolled users to get their impressions of the service.
A user retrieves her package from one of the Pelikan vans.
Limited Run Service
Following preliminary tests at IDEO, we enrolled 10 customers and ran the service in a small area of San Francisco for 4 weeks. In addition to teaching us about what it would mean to do this at a larger scale, we got a chance to interview customers about the experience.
Initial Findings
Our learnings from early testing helped establish a foundation for a larger scale experiment.
Experiment
Criteria
From our early tests we were able to refine what variables we needed to focus on in a larger experiment and redefine our data collection approach accordingly.
Digital Experience Requirements
Leveraging the immediacy of texting with users and augmenting the experience with a web app.
Conditions for a Bigger Experiment
In order to study customer behavior change, we would need more participants and a longer testing duration. We also needed to deliver an experience that felt robust, trustworthy, and believable.
Experiment Variables
We determined that user tradeoffs between time, effort, and money would be the most impactful variables to test in a larger scale experiment. How users spent this “currency” reflected their priorities and behaviors.
Time
Immediate or delayed. Specific or general time window.
Effort
Distance walked. Quantity and size of items.
Money
Price of option relative to other available choices.
The Build
With the insights gathered from the early test, we set out to build a larger scale pilot. To do this we needed to create an experience that was robust enough to endure repeat interactions for an extended period, secure and trustworthy so that it would be believable, and connected so we could collect data on user interactions.
Digital Experience
Based on learnings from the early tests, we had a strong direction for what features our web app would need to support. After getting additional feedback from users, we locked in the design and built it in React.
The app served three main functions:
1) Allow users to schedule deliveries
2) Serve as a key to access the items from the delivery vans
3) Act as a challenge to present different options as part of our experiment
Quick sketches to collaboratively explore possible directions.
Early wireframes to develop the design and get feedback on the experience.
We mapped out how the various parts of the flow would interact with the database where we collecting user behavior data.
Physical Experience
We developed two new vans to add to the fleet, a green “curb side” vehicle with two large compartments and a blue “locker” vehicle with two banks of smaller compartments. Users interact with a touch screen panel on the back of the vehicle and use a QR code to open.
We designed and installed custom cargo compartments and touchscreen kiosks into two transit vans.
Users followed instructions printed on the kiosk and displayed on the touchscreen to interact with the vehicle and scan the QR code from the Pelikan app.
The Raspberry Pi powered touchscreen guided users through the process of retrieving their items.
Service and Experiment
In parallel to the physical and digital builds, we thought through the service experience and the experiment criteria.
Outlining what pieces of data we were capturing at each phase of the journey allowed us to iterate on what experiments we planned to run.
As part of the development of our service and operational model, we used role playing to think through the process.
Once our experiment goals were defined, we created a flow to capture different experiment cohorts and specific moments for research engagement.
We documented our operation into a series of detailed process diagrams that allowed us to deliver a consistent service and train our operations staff.
The Pilot
We enrolled 150 customers and had 50 active users who regularly engaged with our service. By capturing data throughout the experiment on customer behaviors (their actions, selections, habits, etc.), then following up with them through surveys and qualitative interviews, we were able to maximize the amount of learning through the experiment. We ended up delivering over 1000 parcels to our users through the 4 months that we were in operation.
How it Worked
The physical and digital touchpoints came together in an end-to-end service experience for our customers.
Door Service
A courier brings an item to the customer’s front door.
Curbside
Our curbside vehicle drives to the customer’s address and comes outside to retrieve their item.
Mobile Locker
Customer packages are loaded into lockers and the vehicle parks for a window of time, during which they have access to their items.
Running the Pilot
One of the major values of the pilot was the experience of learning through doing. The operational insights and day to day findings shed light on what it might take to scale a similar offering as a real business.
Partnering with the Parkmerced community in San Francisco provided a favorable environment to test the service.
Partnering with the Parkmerced community in San Francisco provided a favorable environment to test the service.
Capturing Interactions
Analytics on the app and sensors and cameras on the vans allowed us to study the nuisance of customer interactions with our service.
Qualitative Research & Data Science
The size of our experiment allowed us to dedicate a lot of attention to each of our participants. This level of engagement with users, coupled with data on their interactions, allowed for deep understanding of customer behavior.
We analyzed behavioral data to identify interesting patterns for further research.
We create snapshots of specific user’s behaviors such as delivery choices, customer support conversations, and data relative to aggregate behavior.
Users reflect on their own data, choices, and patterns in interviews.
We captured qualitative data in the context of the user journey to identify specific moments of impact.
Learnings & Outcomes
User behavior changes with access to better options. Pelikan services changed behaviors and expectations for the customers of Parkmerced. Customers felt that they could order more frequently and with a greater range of online retailers.
User Perspective
Access to Pelikan shifted customer perceptions and behaviors around online shopping.
Notable Data Patterns
We found patterns in customer behavior that provided the foundation for a future moving goods service.
Self-Service Appeal
Self-service options are preferable when users control the conditions of the experience.
Willing to Walk
Customers come from a range of distances within the Parkmerced campus to retrieve their packages from the locker.
Predictable Demand
Pelikan operated daily between 8am and 10pm and within this span there was a noticeable peak in demand between the hours of 4pm to 8pm on weekdays.
Short Windows
When using the locker, customers reserved a 2 to 3 hour window of access to retrieve their item and the majority of customers picked up their package in the first hour.
Impact
The qualitative and quantitative evidence from the pilot, combined with the operational and customer interaction findings we collected, demonstrated to Ford the potential opportunities for future autonomous moving goods services. Our work challenged assumptions held by the company around the desirability and viability of self-service offerings.
Our work informed the go-to-market strategy for a number of initiatives in the moving goods space that Ford is currently evaluating. Specifying an approach that leveraged Ford's unique capabilities, we outlined user experience requirements for a larger scale service based on Pelikan and did an analysis on the business to demonstrate a path to profitability. At the end of the project, we presented findings to the CEO of Ford and the top executives of Ford’s AV LLC business team.