Introduction

In the first part of our special series on discount brokerage rankings and reviews, we take a look at one of the most popular consumer research organizations, J.D. Power and Associates, and their annual survey of self-directed investors’ impressions of Canadian discount brokerages known as the Investor Satisfaction survey.  The results from this survey form the basis for the highly coveted J.D. Power and Associates award for the Canadian discount brokerage industry. As such, both consumers and the discount brokers pay close attention to the results of these surveys.

For consumers trying to choose between the many options in the discount brokerage market, using reviews and data from actual customers is a popular strategy to help make a decision.  With several ratings systems now being provided to self-directed investors, there seems to be some confusing and potentially conflicting messages being communicated about different discount brokerages. Not all reviews are created equally, however, nor do they measure identical features, which is why sometimes the results can seem confusing.

To help demystify the ratings and to give Canadian self-directed investors a better understanding of J.D. Power’s rankings, we’ve taken a look at the following four questions:

1. Who is J.D. Power and Associates?
2. What does the Investor Satisfaction survey measure?
3. How is Investor Satisfaction measured?
4. What does this survey and its ratings mean for consumers?

1.      Who is J.D. Power and Associates?

J.D. Power and Associates is a professional consumer research firm.  They are among the most widely recognized names in marketing research and have a solid reputation for conducting thorough research on the experiences of consumers with brands or products.  Although they are well-known for their work in the auto sector, J.D. Power and Associates (J.D. Power) also conducts surveys across a number of different sectors, including the banking and financial services sectors.  Of particular interest to Canadians looking for a discount brokerage, J.D. Power has been conducting surveys of investor satisfaction with Canadian discount brokerages since 2009. Over that period of time, they have surveyed over 11,500 clients of Canadian discount brokerages asking them about various components of their experience with discount brokerage account providers.  J.D. Power makes their money by selling the detailed results and analysis of their research to major brands so that these brands can better understand the needs and experiences of their customers and, in theory, work to improve those experiences.

2.      What does the Investor Satisfaction survey measure?

The investor satisfaction surveys measures the degree to which self-directed investors find their experiences with a discount brokerage to be satisfying.  More specifically, “satisfaction” is measured along the following 6 factors (in descending order of importance):

1. Interaction
2. Trading Charges and Fees
3. Account Information
4. Account Offerings
5. Information Resources
6. Problem Resolution

While most of the components are fairly self-explanatory, the first category “interaction” refers to how the discount broker interacts with the client.  For example, some discount brokers have branches whereas others are strictly online with a call centre.  The ‘channels’ for interaction are in person, online and on the phone.

3.      How is Investor Satisfaction measured?

At the heart of every survey are the people who are going to be answering the questions.  The team at J.D. Power begins by accessing large databases of individuals that agree to participate in online surveys and have identified themselves as discount brokerage account holders.  Those individuals are then provided with the chance to complete a survey, often in exchange for some kind of reward or incentive for their time and opinion. An example of the type of database that is sampled is the J.D. Power and Associates Power Panel.  No client lists from any of the discount brokerages are used to find individuals, and as a result the data source is considered “third party”.

Large scale surveys, especially of opinions and experiences are complicated to put together.  In order to be meaningful there has to be enough responses to accurately reflect a cross section of clients.  On the other hand, there are some brands that are simply either more popular or have more customers, so some element of balance needs to be ensured so that all brands are comparable.  As such, the investor satisfaction survey requires both a minimum and maximum number of client surveys that need to be filled out each year for each discount brokerage in order for a discount brokerage to be included in the final reporting.  For example, in the 2012 survey, Credential Direct did not have sufficient survey completion to warrant including it in the final rankings and so their results do not appear in the list of discount brokerages reviewed.  Each discount brokerage, depending on its representation in the database, will have an associated number of surveys that must be completed in order to be able to balance the findings so that all companies are comparable.

Once the data is gathered it is then analysed and the rankings across the six categories (mentioned above) result in a final score.  J.D. Power has two components that go into its reporting of investor satisfaction.  The first is a basic numerical score and the second is their proprietary “Power Circle” rating.

The numerical score a discount brokerage gets is based on a 1000 point scale.  An industry average is calculated by taking the average of all of the scores for each discount brokerage. Interestingly (especially for the stats nerds), while the average score for the industry is mentioned, the standard deviation for this score is not, making it difficult to put the “average” into context.  For example, knowing the standard deviation would help to answer whether the scores are all over the place or if they are tightly grouped together. This is important because those considering different discount brokerages would be interested in knowing whether other investors perceive Canadian discount brokerages as essentially similar to one another or if there is a significant advantage to be expected by going with one brand over another.  [note: for a quick and easy explanation of standard deviations and why they matter, click here]

For many people, numbers on their own might seem confusing.  To help consumers with their decision making, J.D. Power opted to use their 5 point “Power Circle Ratings”.  The ratings are scored as follows:

Similar to a “star” rating system, the Power Circles represent categories of performance. In the quest to make things ‘simple’ by using circles, there are certain advantages and disadvantages to keep in mind.  One of the biggest advantages of the circle rating system is speed. As far as making things “easy”, using circles to represent how satisfied investors are is a very quick reference system.  The tiers that the ratings are based on make it reasonable to assume that whoever has placed first has done a better job keeping their clients feeling satisfied than their peers.

When looking further down the list however, the definitions of the ratings really matter. It is clear that there is a lot of leeway given to the bottom 90% of companies i.e. the “better than most,” the “about average” and “the rest.”

4.      What do the survey results and ratings mean for consumers?

For consumers, accessing accurate and reliable information on financial products is vital to making informed choices.  When looking at customer experiences with discount brokerages, it’s important to ask where those opinions are coming from and how they are reported. The results of the J.D. Power investor satisfaction survey are a reflection of consumer opinions and experiences of actual account holders with discount brokerages.  So, for example, when a particular company such as Disnat ranks “highest in investor satisfaction” (as they have for the past four years) according to J.D. Power and Associates, this claim actually means that holders of Disnat accounts rated their experience with Disnat higher than holders of accounts at other discount brokerages – a phrase much trickier to fit into advertisements.

As with any survey or ranking, it is important to understand what is being measured and how it is being measured in order to know how to interpret the survey results properly.  In the case of the J.D. Power and Associates investor satisfaction survey, self-directed investors’ experiences, opinions and perceptions of a discount brokerage are being measured.   Self-directed investors are, however, a diverse bunch which means that the investor satisfaction survey reflects the answers of all kinds of investors, not necessarily the ones that best match your interests or needs.

Perceptions are also challenging to measure, quantify and report accurately.  Reporting of numerical scores, averages or categories of ranking can convey a sense of certainty that readers need to be cautious about.  When evaluating the investor satisfaction survey scoring systems, an important question to ask is “how important is one extra point?” In the case of either a numerical score or a Power Circle, the answer can mean a big difference.

Conclusion

For Canadian consumers, the J.D. Power and Associates Investor Satisfaction survey is unique in that it offers a free, reliable and impartial glimpse at how fellow self-directed investors experience a discount brokerage product.  If you’re reading this article online, chances are you are also part of a large portion of online shoppers that do their research online by using reviews. The bottom line for self-directed investors is that while good decisions start with good research, ultimately the real test of your satisfaction with a product will be when you actually own it.

Read the second part of our series here.