Choosing the correct question to answer is critical to accurately measuring customer satisfaction. A recent article in the Boston Globe is a perfect example of the potential problem.

For years, the Massachusetts Bay Transportation Authority has measured rider satisfaction using a measure of whether the train picks up a rider on time. However, most people actually care more about whether or not they arrive at their destination on time. “These numbers at times do not necessarily reflect what our customers experience or what our customers are seeing,” MBTA deputy general manager Jeff Gonneville said.

The subway system doesn’t run like the commuter rail and buses, which have scheduled arrival times at specific stops. Subways run on timed intervals between trains, known as headway, so the T (as the MBTA is known) is more concerned with whether passengers have to wait longer than expected for the next ride. If the headway interval is every five minutes, and passengers wait five minutes or less, then the T counts that as a win.

But the number doesn’t reflect whether passengers were left on the platform because they couldn’t squeeze onto an overcrowded train; a common occurrence at rush hour on the Orange and Red lines. Nor does it directly account for delays such as sitting in a tunnel behind a disabled train. It also doesn’t capture how bad the wait can get between intervals; whether riders wait for just a couple of extra minutes, or much longer than expected.

One passenger, Patricia Maguire, commutes daily from Dorchester to downtown Boston. She would rate performance at a 75. “If I count my 10 trips a week for the purpose of commuting, I’d say four of them are impacted in some form or fashion,” Maguire said. “How it impacts me directly is the only guideline I have. … How often do I get gummed up by the system?”

Gonneville said the agency wants a new measurement that focuses less on the wait on a platform and more on the overall length of the journey. “We want to move forward with a new metric that focuses on run time because it gets at some of the inconsistencies we may be seeing now,” he said. Implementing that may prove to be challenging since T riders do not tap in and tap out like on some subways –like London’s. Instead, the T will have to estimate the length of time traveling based on ticket purchased and likely destination based on historical usage. However, the measure, even with the uncertainty built in, will be a more accurate indicator of true customer satisfaction because it is measuring the right performance attribute.

We see very similar issues in assessing on-time delivery (OTD) when measuring overall customer satisfaction.  Many companies calculate OTD by subtracting the number of transit days the logistics company provides them from the customer’s desired delivery date, and assume that if the shipment leaves their warehouse on or prior to the calculated shipment date, it will be delivered on time. Unfortunately, our data revealed that the assumption that the estimated transit days represented reality was often untrue, and delivery dates were being missed even when the shipments left the warehouse “on-time”.  

So avoid being oblivious to potential customer issues by being very careful that what is measured is a true representation of what the customer experiences.

Source: Who cares if your train is on time if you’re not? Adam Vaccaro, Boston Globe, April 22, 2019.