In part one of this series, we looked at the background of the Globe and Mail discount brokerage rankings as well as how they’re structured and who they’re targeted towards. In this next part, we take a detailed look at what the discount brokerage rankings are actually measuring and some interesting observations we made about the Canadian discount brokerage industry over time.  Lastly, we provide some important tips to keep in mind when using rankings as part of your product research.

What Do the Discount Brokerage Rankings Measure?

When looking at any ranking or rating, one of the most important questions to be clear on is what the ranking or rating is actually measuring.  In our review of the J.D. Power Investor Satisfaction Survey, we saw that “investor satisfaction” was being measured by six components: interaction, trading charges and fees, account information, account offerings, information resources and problem resolution. By comparison, the Globe and Mail discount brokerage rankings are measuring what Rob Carrick thinks is the “best discount brokerage” for “mainstream” investors.

As we saw in part one, when looking across the last eleven rankings, it appears that the categories that go into defining “the best” discount brokerage are not static. The most stable characteristics of what it means to be “the best” seem to cluster around costs, trading and tools. According to Carrick, the categories that he chooses vary in large part because they are based on a combination of data from reader surveys and his perceptions of what mainstream investors are curious about or would find worthwhile.

Strengths of the Discount Brokerage Rankings

A strength of this approach is that the discount brokerage rankings are somewhat reflective of the mood or sentiment of mainstream Canadian investors.  If investors are curious about certain features, such as commission free ETFs or user experience of a discount brokerage, the rankings have incorporated these kinds of innovations into their structure.  Having looked at a decade of results, it is fair to say that the rankings reflect the pulse of what mainstream investors are exposed to from the discount brokerage industry and hence curious about.

Limitations of the Discount Brokerage Rankings

While Rob Carrick’s opinion is certainly informed by monitoring Canadian discount brokerages for over 14 years, his opinion may not necessarily be shared by other investors, something that readers should keep in mind when doing their research.  The degree to which his opinion can be generalized rests on how accurately the needs of “mainstream investors”, a term that is loosely defined, are captured in the questions he uses to survey discount brokerages and in the process he uses to evaluate their products and services.

A second limitation of the rankings is historical comparability.  Because the criteria have changed as often as they have, it is difficult to compare historical performance of Canadian discount brokerages in a meaningful way.  It may be possible to compare results on costs, trading and tools because of their relative stability as categories however the total scores from year to year are largely incomparable.

Interesting Findings – The Value of History

Taking a step back and looking across the landscape of the Canadian discount brokerage industry as described by the Globe and Mail rankings, we noted some interesting observations.

The first was the relative constancy of the size of the industry over time.  While the players have changed between 2002 and 2012, the numbers of those players has remained steady at about 12 (not counting Interactive Brokers). The constancy could be telling of how many discount brokerages the Canadian marketplace can reasonably support before a company exits either through acquisition or by folding operations.  Earlier this year we saw OptionsXpress elect to exit the Canadian market, passing its accounts over to Virtual Brokers, a signal that perhaps the market was not economical to pursue alongside so many other competitors.

Another interesting observation was who has been on the list the longest. The veterans of this ranking (having appeared in all of the rankings we looked at) include BMO InvestorLine, CIBC Investor’s Edge, Credential Direct, Disnat, National Bank Direct Brokerage, Qtrade, RBC Direct Investing (formerly RBC Action Direct) and TD Waterhouse (now TD Direct Investing).  Of those names, BMO InvestorLine and Qtrade have dominated first place having achieved the top ranking 10 out the past 11 years.  Overall Qtrade has a slight edge over BMO InvestorLine with an average rank of 2.3 versus a rank of 2.4 respectively (see the table below). Interestingly, most of the long appearing discount brokerages listed are “bank owned” discount brokerages and so with the exception of BMO InvestorLine, the Globe and Mail survey is not somewhere that “bank owned” discount brokerages shine.

Company	Average Ranking Score	# of times in ranking between 2002 – 2012
	2.3	11
	2.4	11
	4.5	4
	4.6	11
	5.3	3
	5.5	11
	6.2	11
	7.3	11
	9.3	6
	9.6	11
	9.8	11
	11.6	9
	N/A	N/A
	N/A	N/A
	N/A	N/A

At the bottom end of the list, HSBC InvestDirect and National Bank Direct Brokerage have consistently performed very poorly in these rankings. In fact, when we took the average ranking of discount brokerages between 2002 and 2012, the poorest performers (from worst to best) were HSBC InvestDirect, Disnat (classic) and National Bank Direct Brokerage.  This was a very interesting finding as both Disnat and National Bank Direct Brokerage have consistently placed at or near the top of the J.D. Power and Associates Investor Satisfaction Survey for the past four years.

By looking at the historical data, what clearly stands out is the extent to which discount brokerages prioritize evolving with the times.  Looking across the ranking commentaries, it is evident that certain companies are much more responsive to change than others. Responsiveness to change is something that consumers ought to pay attention to because who they choose to open an account with today is also who they’ll be doing business with in the future.

Tips on Using Rankings in Research

Rankings such as these are important reputation building tools for discount brokerages and viewed as such, those who place first in a category will often use this achievement as part of their marketing strategy.  For example, this past year (2012) Virtual Brokers placed first overall (and as such “the best discount broker” overall) however BMO InvestorLine, who came in third overall, reported themselves to be “the top bank-owned” online brokerage.  While both are true, the claims sound confusingly similar and thus for consumers, it is important to look closely at rankings that companies use as part of their marketing.

Another tip for consumers using the Globe and Mail discount broker rankings for their research is to understand that their own research may provide different results.  The survey items that go into the rankings are partially explained but not to the point where individuals can see exactly how or why scores came to be.  The way it stands now, the results of the Globe and Mail discount brokerage rankings reflect the experiences and opinion of a particular individual and the experiences/opinions of others are likely to vary.

Summary

Looking back across the past eleven Globe and Mail Canadian discount brokerage rankings has provided some interesting perspectives on the Canadian discount brokerage industry as well as on the rankings themselves.

The Globe and Mail’s rankings reflect what the author of the rankings, Rob Carrick, believes to be important components for mainstream do-it-yourself investors to consider when researching discount brokerages.  As a reflection of what’s current in the discount brokerage industry and what mainstream investors are talking about, the rankings offer a great deal of value. Importantly, though, it is difficult to generalize his opinion and experiences to that of the wider investing public and it is difficult to compare the ranking scores across time, which are two important limitations.

That said, Rob Carrick is particularly experienced in covering the Canadian discount brokerage sector and as such, provides an informed opinion of what investors can expect to encounter when comparing discount brokerages.  If or when an investor is considering a discount brokerage, the Globe and Mail rankings can provide investors with important insights into the products and services being offered by Canadian discount brokerages while the follow-up articles associated with the rankings can add additional context to the data in the rankings themselves.