In part 1 of this series, we took a brief look at how self-directed investors typically access information about discount brokerage customer service and introduced Dalbar Canada’s research in this area.  Specifically, Dalbar’s Direct Brokerage Sales Effectiveness (DBSE) program measures the performance of client service teams at discount brokerages.

In this part of our series, we take a closer look at the evaluation itself to see how Dalbar measures quality customer service, how the evaluations are conducted and what the implications are for self-directed investors when comparing customer service at discount brokerages.

Knowing what counts

Given the number of evaluations being conducted to try and find ‘the best’ Canadian discount brokerage, it is increasingly important that self-directed investors be clear on exactly how a measurement of a discount brokerage is structured.  To avoid falling victim to clever marketing, whenever Canadian investors see a discount brokerage ranking, the following three questions are worth keeping in mind:

1. What is being measured (ranked)?
2. How was it measured?
3. Does the measure actually measure what it says it does?

In the case of Dalbar Canada’s Direct Brokerage Service Award, what is being measured is “quality” of the customer experience when interacting with a particular discount brokerage’s customer service agents.  Of course, how “quality” is defined can be highly subjective so readers should keep this in mind when considering the explanations below.

A framework for quality customer service

To provide some structure around the idea of customer experience Dalbar surveyed (and periodically continues to survey) Canadian investors regarding what they believe to be important components of a quality customer service experience.  Using that consumer feedback, Dalbar has defined a “customer experience” according to the following five criteria with each component having its own weight/importance (see figure below):

1. Expertise
2. Accommodation
3. Professionalism
4. Sales Effectiveness
5. Chat/Email

Even though there is still a degree of subjectivity in terms of how the weights are assigned, having structured and transparent criteria allows others to determine whether or not the criteria and the associated weights make sense to them.

How it all comes together

Over the span of about a year, Dalbar analysts carefully review the quality of discount brokerage customer service by using data from actual telephone and email interactions with account holders.  They obtain this information from clients enlisted in a “mystery shopper” style program so that Dalbar’s evaluations capture as authentic an interaction experience as possible.  Questions or requests can span a range of common customer service situations including everything from questions about specific services to obtaining a stock quote by phone.

The analysis is performed using custom software specifically designed to ensure that evaluations are conducted consistently from one encounter to the next. Once the data is analyzed, discount brokerages are compared with one another on their performance across the categories mentioned above.  The company that scores the highest on the evaluation is then awarded the Direct Brokerage Service Award.

The service experiences that Dalbar tests cover the most common requests made by investors to their discount brokerage.  Often, this is precisely the kind of information potential clients are looking to find out from an unbiased source before making the decision to choose one discount brokerage over another.

Customer Service Evaluation Results from Canadian Discount Brokerages in 2012

Dalbar Canada has exclusively provided SparxTrading.com with the average score from their 2012 evaluation along with where all Canadian discount brokerages tested landed relative to this industry average.  The following graphic describes the results obtained:

The winner from the past year was HSBC Invest Direct scoring 78.54 out of 100. Interestingly, RBC Direct Investing, which had won this award for 5 consecutive years prior to HSBC Invest Direct, was a close second at 77.17. Both BMO InvestorLine and TD Waterhouse Discount Brokerage also managed to score above average.

As the diagram illustrates, the majority of discount brokerages covered here are subsidiaries of larger Canadian financial institutions. Interestingly, in terms of service experience two independent discount brokerages (Qtrade and Questrade) as well as several major bank-owned brands scored under the average, suggesting that size doesn’t necessarily translate into better service.

Does Dalbar’s Customer Service Evaluation Measure What it Claims to?

For the most part, the components that constitute the Direct Brokerage Sales Effectiveness program reasonably approximate the kinds of things one would expect to happen when calling up a discount brokerage and interacting with a staff member on the other end.

It is reasonable to expect that most clients would want to have their issue(s) properly understood and addressed.  It also seems reasonable that being treated as a customer (i.e. being addressed by name) rather than a number would make for a more pleasant interaction.  Exactly how important each of these components might be likely varies between people, however the criteria and the method (multiple measurements conducted throughout the year) to measure them seem appropriate.

Conflicting Opinions on Discount Brokerage Service

While the process used to sample and measure customer service experience is highly structured, there are conflicting data points from evaluations conducted by Surviscor, another organization which conducts Canadian discount brokerage rankings, which bear mentioning.  For example, RBC Direct Investing had been the winner of the Dalbar Direct Brokerage Service Award for five years straight (from 2007 to 2011) however in the 2012 Surviscor evaluation of direct brokerage response times, RBC Direct Investing had response times by email of greater than 16 hours.  Differences in methodology, scoring and definitions of what a “complete” answer from a customer service agent means could explain the discrepancy.  Also, while email response times and telephone customer experience aren’t necessarily related, they do reflect two different means by which customers interact with their discount brokerage.

The fact that there is such a large discrepancy between evaluations highlights why it is important to find out how measurements are conducted and ask what exactly is being measured. As a case in point, according to Dalbar’s research, individuals sometimes don’t mind waiting a bit longer to get an answer so long as the answer that they are getting actually addresses their needs satisfactorily.  Thus, how long a response takes may not trump the value of having the desired outcome from the interaction.

Limitations

While the data and analysis of the customer service quality at Canadian discount brokerages appears to be reliable, one of the biggest drawbacks of the Dalbar evaluations is that only the winner and/or top candidates are typically reported.  Other evaluations of discount brokerages, such as J.D. Power’s Investor Satisfaction Survey or the Globe and Mail Discount Brokerage Rankings report the results of the entire sector, as well as their method of determining the best discount brokerage.

Because the results are not more widely published, the Dalbar award, while clearly well earned, does not allow for consumers to contextualize how much better or worse the winner of the award performed relative to its peers.  As such, even though the award is widely advertised alongside discount brokerages such as RBC Direct Investing and HSBC Invest Direct, many self-directed investors may simply be unaware of what it took to earn the top spot.

When asked whether or not Dalbar would be publishing additional information regarding the results of their evaluations, they stated that it is something they are considering, however, there is no firm commitment or timeframe around it happening.

Conclusion

Even in the “self-serve” online brokerage world, interactions with a discount brokerage’s customer service still play an important role in the process of getting questions answered and requests handled for many self-directed investors.  Often it is these instances that form lasting impressions on consumers and the ones they are sure to pass along to others when asked about their personal experiences with a particular discount brokerage.

The Dalbar Direct Brokerage Service Award is a unique assessment within the landscape of discount brokerage rankings and evaluations. Its focus, while much narrower than the other awards, is probably its greatest strength because it is able to provide a much more detailed understanding of the interface between client and provider.

For self-directed investors who place a premium on service experience, the Dalbar award (and/or evaluation score) is worth considering as part of an evaluation of a potential service provider. Even for those that may not put a high level of importance on customer service, it is still nice to know who out there is working harder to treat their clients as people.