YouTube on TV On-Device Survey

 

 

Understanding Why Users Don’t Sign In to the YouTube app on TV

Role: UX Researcher for YouTube

UX Research Project Lead:

Eileen Ellis

Product Team:

Haley Burns,

Engineering Team:

Steve Warner,

Data Engineering Team:

Peter Situ,

Design Team:

Matthew Anderson,

Clients:

YouTube was a carsharing service created by Penske Corporation that operated publicly in Washington D.C. and Arlington, Virginia from October 2019 to April 2020. Penske Dash offered A to B carsharing with the option for members to park and end rentals on the streets and in partner lots/garages in the Washington D.C. and Arlington areas. The service was closed after 7 months of operation due to the change in the economy amid the COVID-19 virus.

www.youtube.com

Ridecell engaged with Penske Dash to develop a B2C carsharing business serving Washington D.C. and Arlington, Virginia. The carsharing model was new to the US, as it combined the free-floating model where a user can park at any meter or free parking spot (considering normal parking restrictions) as well as within selected Colonial parking garages and lots in the DC area.

I was involved with the project from April 2019 to April 2020, from pre-launch and pre-SOW signing where the selected partner garages had not yet been identified, through the beta and public launches of the service, and post-launch as new features such as integrations with companies offering parking lot barrier arm technologies were being tested for public use. The Penske Dash service was officially closed down in April 2020, due to the effects of the COVID-19 virus.


Overall UX Research Objectives

The objectives of UX Research for the Penske Dash service for the duration of the project (1 year) covered 2 main phases:

UX Research Pre-Launch (April 2019- October 2019)

  1. Analysis of in-market competitors. Understand how the Penske Dash service would operate with the partner parking lots and garages by testing out already existing competitor services

  2. Suggested parking locations for Penske Dash. For the client, offer suggestions as to the type of garages/lots they should include in their service, which they should avoid, and any recommendations to improve the experience for users outside of the app

  3. Mapping of current carsharing user journey in-market. For Ridecell, understand the possible user experience of the Penske Dash service and identify pain points and opportunity points that would need to be addressed

  4. Usability testing of Penske Dash test app (Friends and Family beta testers) (not shown)

  5. New feature development for Ridecell

  6. Usability and prototype testing of the Penske Dash app (internal Ridecell team testing) (not shown)

UX Research Post-Launch (October 2020- April 2020)

  1. Usability and prototype testing of the Penske Dash app and service (user testing)

    1. Identify points at which there are pain points, or a reluctance to enter information, if any during the registration process within the Ridecell app

    2. Understand typical user behavior and views about condition and damage reporting of carsharing and traditional rental vehicles according to user segment

      • Identify user fears, motivations, and expectations about damage and condition reporting

      • Determine terminology and categorization recommendations

      • Identify pain points, positive points, and needs

      • Receive possible design and functionality recommendations

    3. Understand general usability and opinions on booking a rental and navigating to the vehicle with the Ridecell app

    4. Test ability for user to locate the vehicle when the vehicle is parked in an underground garage (GPS accuracy and in-app directions)

    5. Understand other topics users would be open to self-report, mainly focused on damage and condition reporting but may include the vehicle location for the next user, etc.

    6. Understand the begin and end rental flows and overall experience

    7. Test scenarios involving after-hours garage access codes and QR code retrieval for exiting garage


Pre-Launch: Research Plan and Methods

To do this, our research team needed to do in-market immersions in Washington D.C. and Arlington, Virginia to gain a preliminary understanding of the environment of our target users (city and suburbs) and the different Colonial parking garage, lot, and structure types. As there was a limited budget for this research project, no users could be interviewed, so it was important for the team to try to empathize with different types of user personas and go through a simulation of their experience, by testing the services of Penske Dash’s existing in-market competitors.

After the in-market immersions and over the following months, I worked with the Penske Dash team to conduct testing and evaluation of the Penske Dash service with a group of beta testers prior to the public launch. As the beta testing findings were mostly related to the technical functionality of the Ridecell app and Penske Dash service, as well as the experience of using the Volkswagen vehicles themselves, these findings will remain confidential and will not be expressed here.

Pre-Launch: Research Trip

Research Prior to Travel

  • The terminology that would be used to categorize the different parking types for both Ridecell and the client was set as one of the first steps in planning the research. The nomenclature was selected by researching architecture white papers and construction and parking company websites.

  • The Ridecell Research Team identified which parking lots/ structures/ garages to visit on the ground based on their locations as part of the urban landscape, type, carsharing service rental availability, and team and client schedules.

  • Current pain points users face was researched by looking at Yelp reviews of specific parking garages/ lots/ structures, reviews displayed on the Colonial parking website, and cold calling the garages to speak with attendants.

  • Knowledge was gained about existing competitors like Zipcar, Getaround, and Maven by exploring their official websites. Since the Penske Dash service was not yet in the market, Zipcars and Getaround vehicles located in selected Colonial parking garages/ lots/ structures were reserved for testing purposes.

  • A cognitive walkthrough of the user journey was conducted to set up the research recording structure in Airtable. This was important as the findings needed to be recorded on foot or in the vehicle while parked and driving, so the mode of recording had to fit those research team needs.

Research In-Market

  • In-market competitor testing and evaluation was conducted. In addition to the carsharing services tested during the immersive research, the Ridecell Research Team also tried Car2Go and Free2Move to understand the already existing freefloating models available to users in Washington DC and Arlington, VA.

  • Think aloud protocol was used by the Ridecell Research Team to narrate thought process across different steps of the user journey (our team did not have the budget to speak to actual carsharing users in-market at this point).

  • Intercept interviews with Colonial garage attendants and valet staff were conducted to understand how they worked with the already available carsharing services at their parking location, and what positive points, pain points, and needs their customers had in using those services.

 

 

Post-Launch: Research Plan and Methods

The post-launch research for Penske Dash was the first at Ridecell to include non-internal stakeholders, i.e. people who are consumer users of Penske Dash or competitor services. It was important to gain the consumer user’s understanding not just from a product creation standpoint, but also as the first formal user researcher hired at Ridecell, to get team members at Ridecell to understand the importance of using user research to make business and product decisions.

Post-Launch: Research Trip

Research Prior to Travel

  • The Penske Dash Launch Team Manager and Customer Success Manager were interviewed to learn what feedback had already been received from the client pertaining to the research subject.

  • A remote workshop with the Penske Dash client team was conducted to:

    • Understand Penske Dash’s current damage reporting process including types of damage (including cleaning), how Penske Dash does reporting internally, how they manage and resolve issues, what data needs to be stored, what information they need to access periodically (analytics), and applications for different types of services.

    • Review information Ridecell has already received from Penske Dash about damage reporting, to receive clarification, if needed.

    • Bring up use cases and test hypotheses with the customer according to the conversation.

    • Understand Penske Dash goals for damage and condition reporting, including what they are aiming to do in the future i.e. plans or processes that are already underway.

    • Prioritize Pain Points/Proposed Changes brought up through the workshop.

    • Develop criteria about what Penske Dash needs Ridecell to consider when designing the solution.

  • Stakeholder interviews were conducted with different Ridecell internal teams including Product, Design, Engineering, Launch, and Customer Success, to understand what they would like to learn from the user research trip and to gain buy-in for the process and results.

  • Cognitive walkthroughs were conducted with the main Penske Dash product designer to locate gaps and pain point areas in the flows the research team would test, according to different scenarios such as: different travel methods to the garage location, during and outside of garage business hours, whether the user was registered or not, whether the user was signed in or not, if the user was away from the vehicle being reserved vs. at the vehicle, etc.

  • Since Ridecell was not allowed to use the actual Penske Dash consumer app and service (vehicles, Colonial partner parking garages) for testing, I needed to coordinate with engineering team to reserve and learn how to set up our in-vehicle carsharing hardware product, identify Ridecell white label apps to test with, reserve a QA environment to ensure that the service would not be disrupted while in use, and gain buy in and support when the research team traveled onsite in case we needed to troubleshoot stability issues.

  • Other scoping and research was conducted to set up the research trip, including:

    • Scoping and managing the research budget including: travel costs, recruiter costs, respondent incentives, vehicle rental, office space rental, parking garage payment.

    • Worked with the Ridecell legal, insurance, and procurement teams to ensure regulatory, client, and company compliance.

    • Recruiting Ridecell team members to support the research work in-field.

    • Identifying, selecting, and managing research recruiting vendors, and developing user research assets such as screeners, discussion guides, legal documents (NDA and recording releases), and incentive payment receipts.

    • Other research and setup for the research project, including renting a similar vehicle through Turo, selecting a comparative parking garage in an area near to a Penske Dash partner garage, reserving a nearby Breather office space for part of the testing.

Research In-Market

  • Conducted 17 user research in-person interviews with in-field usability research tests, at 2 hours per respondent.

    • Respondents were limited to the markets of Penske Dash in Washington D.C. and Arlington, Virginia, and were segmented according to 4 main types:

      • Intender (previous car renter from a traditional car rental agency like Hertz, Enterprise, etc., but not a current carsharing user)

      • Current Penske Dash Users

      • Competitor Users (does not use client product Penske Dash, but does use competitive products like Car2Go, Free2Move, Zipcar, etc.)

      • Former Penske Dash Users (have used it at least one time before but have not used it in the past month or longer)

    • The research structure was as follows:

      • User Research Interview. The respondents met the Ridecell user research team at the Breather office location and were interviewed to gain insights about their background, vehicle rental and carsharing experience, etc.

      • Simulated Lab Usability Testing including Task-Based Analysis. The respondents were asked to complete a series of tasks while in the office space, including registering for the service, reserving the vehicle, and reporting damage on the vehicle via the in-app damage reporting flow. Respondents were also prompted to gain feedback on specific features, placement of buttons, terminology, etc.

      • In-Field Usability Testing including Task-Based Analysis. Respondents were asked to locate the test vehicle on foot using the Ridecell white label app. The test vehicle was parked in a nearby parking garage that mimicked the Colonial partner parking garages with barrier arms. During this process, the Ridecell research team asked the respondent a list of questions, and asked them to complete tasks including beginning and ending the rental.


Research Findings

The biggest impacts from this phase of the research pertained to the categorization, prioritization, and troubleshooting different types of Colonial parking lots/ garages/ structures for the initial launch phase and subsequent rollouts of the Penske Dash service. For Ridecell, the research findings and implications impacted the design of solutions and the reorganization of the product roadmap to incorporate the development of the new features or manipulation of the old ones.

Offering the Service in High Vehicle and Foot Traffic Areas May Not Be Ideal

While it seemed like offering the service in high traffic areas close to multiple modes of public transportation made sense, our research found that these areas in D.C. were often hectic due to the urban design of the area, the types of parking that were available in the area, and the high foot and vehicle traffic. Therefore, there was a high chance that the service may not be utilized if located in those areas.

User Experience Implications

For a user with little driving experience, who may be using the service for the first time, or who may be familiar with the locality on foot but not necessarily via driving, having the service in a high traffic location may cause stress and result in a negative view of the Penske Dash service and brand.

Business Implications

Since the Penske Dash partnership with Colonial entailed that Penske Dash would need to pay for the parking spaces it uses at Colonial locations, it was important that locations with a high probability of utilization were selected, especially during the first phases of the service launch. If the service was launched in a low utilization area, the money paid to Colonial for the ability to park there would be a sunk cost.

 

 

Locating the Vehicle is One of the Most Important Parts of the User Journey

Through the research, the team found that one of the biggest pain points the user may encounter upon use of the Penske Dash service was having to locate the vehicle when parked in a parking lot/ garage/ structure.

An example of this was with the Union Station Parking Garage (shown above). It took the research team so long to locate the Getaround vehicle that was reserved for the research test (it took over 30 minutes), that the team was almost forced to cancel the reservation. Despite the fact that Union Station offered both Maven and Getaround services, complications arose because there were multiple parking garages, multiple entrances to the parking garage containing the Getaround cars, lack of knowledge from the staff that the carsharing service even existed in their facilities, lack of signage leading to and within the garage, etc. In the end, it was decided that the Union Square Parking Garage should be excluded from the Penske Dash service due to the difficulty in being able to find the vehicle.

User Experience Implications

By clearly communicating the positive points and pitfalls of competitor services across different types of Colonial parking, we were able to give details to the client, outside of the Ridecell products, as to some recommendations to make it easier for users to get over the initial hurdle of finding their car.

Business Implications

Since the business revenue is driven by vehicle rentals, if a vehicle is not rented because it is difficult to locate, the ROI on that particular vehicle would be low. At worst, the underutilized vehicle may in fact cause the service to lose money as no money would be generated to offset costs, and more costs would need to be incurred to have someone come to rebalance the vehicle (move the vehicle to a more desirable location) and/or have to pay parking fines as there are often limits on how long a vehicle can be parked at a particular location on the street. By putting focus on these initial steps of the user journey, the possibility of low vehicle utilization because the renter simply couldn’t find their car would be greatly reduced.

 

 

The Driving and Parking Experience Should be Considered

In addition to the physical location of the Colonial parking garage/ lot/ structure, the user experience of actually driving the car in and out of the space, as well as finding where to park, needed to be considered when considering partner locations for launch. This not only means that the size and placement of the parking spot needed to be evaluated, but also the surrounding architecture of the parking space, wayfinding for driving within and exiting/entering the garage, and instructions and availability of support staff when confusion arises.

User Experience Implications

A feeling of intimidation may arise from just the prospect of conducting difficult driving maneuvers to get in and out of tight parking spaces, and may cause a user to cancel the reservation. In the future, a user with minimal driving experience may be intimidated with the prospect of driving out of or parking within a tight parking spot and so this may impact their decision to rent vehicles in specific parking lots/ garages/ structures with these features.

For example, during the research trip, one Colonial garage that was evaluated had parking spots aligned in such a way that it took a 5-point turn just to drive out of the spot, even though the driver was highly experienced. The vehicle that was rented had a large dent in the side from a previous driver who had damaged the car trying to get out of the spot, which had a large column to one side (shown above).

Business Implications

Since each parking space had to be paid for by Penske Dash to Colonial, it was important that the spaces be used not to park a single vehicle for long lengths of time (which pointed to low utilization) but for numerous shorter lengths of time (high utilization).

Additionally, a difficult parking and driving situation meant that the Penske Dash vehicle was more likely to end up damage, thereby increasing the maintenance cost of the vehicle if it was repaired, and damaging the brand image if left unrepaired but still available for public use.

 
 
 

The Registration Flow is OK and Doesn’t Need to be Revamped

6 months before the Penske Dash post-launch research project with consumers, I conducted internal Ridecell interviews where I spoke to 32 people from every department within Ridecell to scope out potential research work for the UX research roadmap for the following year. A common theme that came up during these interviews was the idea that the Registration flow of our white label mobile app (and therefore the registration flows for all of our consumer user-facing mobile apps) needed to be updated in a big way. I decided to take the opportunity of the consumer user research trip to test the usability of the Registration flow and understand what target consumers thought about aspects like terminology, features, etc.

We found that aside from certain terms that needed to be changed due to confusion about their meanings, and concerns about privacy and verification (as driver’s licenses and faces needed to be scanned, and driver information needed to be verified by a third party), not much actually needed to be changed about the registration process.

User Experience Implications

Many respondents we took through the Registration flow found the process easy, but a couple of people brought up concerns pertaining to trust, privacy, and responsibility. For example, some terms, like “verification process,” led respondents to question aspects of the service like who exactly was doing the verification, what kinds of information the entity was verifying, verification criteria, and the waiting period to be able to use the Penske Dash Service.

When prompted during the research interviews however, we were able to uncover easy remedies to the problems, like making sure that the verification process steps and information being verified was explained clearly, plainly, and in larger text. Changes like these help build trust in the Penske Dash service, which would help with sign up numbers.


Research Impacts

Company refocus

As the client was wary in terms of anything that would negatively impact the brand, especially in terms of car damage and the overall user experience, the findings from the research impacted the selection of specific parking garages, especially for use cases like when overflow parking is needed.

Image Credit: Penske Corp.

Image Credit: Penske Corp.

On-Site Penske Dash Parking Signage

Designated parking spots and clear Penske Dash signage at each spot were being discussed, but the need for them was made clear through the on-the-ground research and creation of deliverables such as this journey map and presentation.

New Penske Dash App Feature: Parking Pin

For Ridecell, the research findings meant that new features needed to be developed to aid in the wayfinding of the vehicle.

A Parking Pin was created to indicate the location of the particular parking lot where the vehicle was located. On the technical side, since the service was not station-based (meaning that the vehicles were not associated with one Colonial garage/lot/structure but can be parked at any Colonial parking locations), the Ridecell team was limited in their ability to show the vehicle pin as “absorbed” into the parking pin at this time.

In-App Parking Information Sheet

As the comfort level and perceived convenience of different parking lots/garages/structures is largely subjective, the way the Ridecell team addressed these issues was through the provision of more detail in terms of parking location information. This way, the user would have key pieces of information they would need when it came time to find a place to park.

These included:

  • Parking location photos on the Parking Information Sheet

  • Wayfinding information with directions to the lot/ garage/ structure entrance

  • Icons indicating parking features such as:

    • Hours with Attendant or Valet presence

    • Wheelchair Accessibility

    • After Hours Access

  • Usability Instrumentation


Role-Specific Tasks

Photo Apr 29, 10 32 42 AM.jpg

UX Research Project Leader

I led UX Research for the Penske Dash service, which included the scoping and planning of the research projects, Ridecell team training, research findings synthesis, research materials and deliverables creation, findings communication, and represented the user in product, business, and design meetings.

Research Project Creation and Planning

I worked with the Customer Success Manager and Launch Manager to identify the scope of the Colonial parking lots/ garages/ structures, as this information came directly from the client. I identified the variations of the different parking types and cross-checked them with existing competitor carsharing services that would be available during the times of our research trip to create the research itinerary. I also created the in-field data collection and filing method.

Research Team Training

I had created a research training document that included clear guidelines as to the roles that each person on the team had to fill, what needed to be recorded and how, what the itinerary for the week was, and where to upload the research findings files. The people on the team were myself, a Customer Success Manager, and Launch Manager. The other team members were not trained in design research, so more emphasis was put on how to think through the experience of the competitor services from the viewpoint of multiple personas.

IMG_8547 copy.JPG

In-Field Research and Contextual Interviews

I led the team in conducting in-field research where we evaluated carsharing competitor services at different Colonial parking lots/ garages/ structures and did intercept studies with Colonial parking attendants and valets while on duty at their parking locations.