Background

The primary goal of the Massachusetts Bay Transportation Authority (MBTA) is to provide reliable, safe transportation to everyone who lives within eastern Massachusetts and parts of Rhode Island. During this project, my focus was on the MBTA website, mbta.com. The MBTA website garners about 14 million unique visits every month.

In Spring 2021, the MBTA Website team hadn’t made any significant updates to the site's information architecture (IA) and navigation since 2017. It was designed based on rider behavior from that time, and we hypothesized that on some level, site usage had changed since then. Through expert reviews from third parties, we learned that navigating on a mobile device might be challenging for our riders. Therefore, we decided to investigate further into this budding hypothesis to see if there were any improvements we could make. We want to ensure that the website's navigation is easy and valuable for all our riders.

Outcomes

some Research takeaways

• Riders found it more difficult to find key areas of the website on a mobile device than on a desktop.

  • Some areas that were particularly hard for riders to find were station info, service changes, improvements, and public meetings

• Riders tend to rely on the homepage for primary navigation and use the header or footer navigation as a fallback.

post-launch

 This project is slated to be launched no later than the end of January 2022.

Some metrics we’re hoping to change with this new experience:

• Increase the number of selections in the menu after it is opened

• Increase pageviews to page with new links in header + footer nav

• Decrease time on the homepage when rider interacts with the menu

• Decrease pageviews or sessions with multiple open menu clicks

My Role

Sole product designer leading user research, ideation, and the final visual design.

Contributed to product strategy, scope, and technical implementation.

The rest of the team was comprised of one product manager, one project manager, three software engineers, and three content writers.

Our research exposed how the biggest area for improvement was how riders navigate on a mobile device. We updated the mobile nav to be in parity with the desktop experience. Having all the same links available in a fast and performant manner. We also added links that existed within the site structure but were not reflected in the navigation. These added links represented tools or content that are important for all riders to have equal access to. We renamed several links/categories and reorganized their groups to better reflect our rider’s expectations of what info belong together.

Discover

Content Audit & Analytics Review

I conducted a content analysis to evaluate the existing site structure and content distribution from a high level. I didn’t want to work off of assumptions of what the site consists of. It was important to me to identify the complete breadth and depth of the site before diving into any structural changes. I aimed to answer the following questions, inspired by Lisa Maria Martin’s Everyday Information Architecture:

• How much content is on the site?

• What kind of content is it?

• How is the site structured?

• How effective is the content?

• How is the content managed?

I also partnered with my product management counterpart to investigate several user behavior questions. We wanted to be able to describe how riders use navigation currently and how it compares to search usage. In addition, we identified the most popular search queries.

Usability + Tree Testing

This study aimed to evaluate the current IA and Navigation of the website against critical tasks. We conducted five tree tests and ten usability tests with the following goals in mind:

• Establish a baseline of how riders interpret the current site structure

• Identify any friction points when riders perform everyday tasks.

• Establish baseline performance and satisfaction levels of the user interface for future usability evaluations.

I partnered with my content writing and engineering counterparts on notetaking for all testing sessions.

Open Card Sort

We ran a series of remote, moderated card sorting sessions with five public transit riders. Participants were asked to put 50 items from a list into logical groupings they would expect to find on a public transit website. They then labeled each group of terms. Participants were not required to sort all the cards. The cards were also randomized for each participant.

high-level takeaway

Out of the 50 items participants were asked to put into groups, 13 distinct groups emerged. Of the 13, only 6 had any overlap amongst participants. There was a definite trend to group modes of travel together (e.g., Subway, Bus) and group items about the agency (i.e., Leadership, Financials). They were most unsure about Gifts, Commuter Rail Positive Control, FMCB, The RIDE, Fare Transformation, CharlieCard, and Add Value to CharlieCard.

There was a significant amount of confusion with the Civil Rights and Climate Change links. Particularly Climate Change, participants, weren’t sure if they’d find active projects by the T or more of a generic mission statement.

Competitive Analysis

The goal of this analysis was to identify any significant trends in how transit agencies present their website navigation. By gaining a clear picture of how other agencies structure their site we have a stronger understanding of what our riders expect from the MBTA website. Ultimately, the MBTA doesn’t significantly differ from many of the navigation patterns used amongst transit agencies across the US and some in Europe.

some high-level takeaways

We found that:

• Generally, the same number of links in the header exists across device type

• Very few agencies had a standalone page for submitting a search query.

• Half of the agencies display their contact numbers in the footer

• Vast majority of agencies don’t have fare prices shown in the header

• Half of the agencies don’t use an overflow (i.e., “More”) menu on desktop

Ideate

Navigation Structure Concepts

After discovery research wrapped up, I partnered with our staff content writers to develop some initial navigation structure concepts. We individually created our own structures and came back to together to discuss the pro/cons across them. The research informed all of our concepts but they are still distinct in many ways. Through discussion, we were able to distill our concepts into one single concept for refinement testing.

Refine

Testing Concepts

With our single test concept, we ran another set of usability and tree testing. Since we had already had a baseline from our initial testing, the goal here was to identify if our changes improved the usability of the navigation. I wasn’t able to test with actual MBTA riders in the initial discovery research phase so I made a point to correct that this time around. Instead of recruiting riders from Userlytics, we created a popup for mbta.com to recruit our riders in context. It was a combination of a popup and intake form over the course of a week that gained up 248 MBTA riders to reach out to for research.

Retrospective

Accessibility must be prioritized
We had hoped to recruit and test out the current website and potential solutions with folks who depend on assistive technology, like screen readers and screen zoom. By understanding this group's unique pain points, we could have simultaneously addressed issues that affect the most underserved riders and improved the experience for everyone else. Unfortunately, we did not have the infrastructure for remote testing with these individuals. This highlighted a significant issue in our research capabilities which we will address by improving our recruitment methods and remote testing tools.

Our goals lacked clarity to start
This project was a case of “just go out there and find problems we can fix.” I wish I had worked with my product partners more to establish some concrete goals and success criteria from the beginning. The research we conducted highlighted some areas for improvement. Still, since we lacked focused goals, the research lasted longer than anticipated and left less time for exploring solutions.

Research with both MBTA riders and general public transit riders
Prior to the pandemic, research with riders typically occurred in person at T stations or bus stops. Since we were all remote, we had no immediate access to our riders. Therefore, for the discovery research we opted to use our remote testing tools user panel where we could at least recruit public transit riders that weren’t local to the area. Our results gave us a clear picture of how visitors to the MBTA would navigate the site. However, when we ran the refinement testing with actual MBTA riders, we noticed slightly different behavior and expectations. Being a local rider with some knowledge of the system significantly impacts how you look for information on the site.