Ratings, Reviews & ROI: How Leading Retailers Use
Customer Word of Mouth in Marketing and Merchandising

Brant Barton

Business Development of Bazaarvoice


Online word of mouth takes many forms, from individual blog posts to threaded message board conversations, but one particular form is quickly emerging as the new consumer-generated media of choice for leading online retailers and e-commerce sites focused on extracting both immediate and long-term ROI from their marketing efforts: customer ratings and reviews. Highly successful retailers like CompUSA, PETCO, and Burpee are leveraging customer ratings and reviews to enhance their existing online marketing, merchandising, and brand building efforts, including search marketing, email marketing, online display advertising, data-driven merchandising, blogs, and RSS. These retailers and others have discovered that customer ratings and reviews are a highly desirable and effective form of word of mouth for multiple reasons. Ratings and reviews are highly focused on the objects of the purchase task, frequently sought out by consumers, and highly measurable.

While e-commerce pioneers such as Amazon.com and eBay embraced customer ratings of products, marketplace vendors, and auction participants early in their history, the multi-channel brands currently driving the mainstreaming of consumer e-commerce have been slow to adopt ratings and reviews even as a consumer research tool much less a marketing tactic. In a survey of 137 retailers conducted by Forrester Research and sponsored by Shop.org, a division of the National Retail Federation, just 26% of participating retailers offered customer ratings and reviews on their web sites but 96% of those who did rated the “merchandising tactic” as effective or very effective at increasing online conversion rates. Recent research on the reported benefits of customer ratings and reviews, such as Shop.org’s annual State of Retail Online report and eVOC Insights’ syndicated retail report, Ratings, Reviews and the Customer Decision Process: Amazon vs. Best Buy vs. Circuit City vs. Wal-Mart, as well as the regular research findings of the Word of Mouth Marketing Association (WOMMA) and its member organizations, have elevated the visibility and status of customer ratings and reviews as a legitimate consumer marketing tool.

This whitepaper presents case studies from three clients of Bazaarvoice, a provider of online customer ratings and reviews functionality and supporting services. All three companies launched customer ratings and reviews on their web sites in late 2005 and have observed measurable business impact from the investment. Bazaarvoice worked with each company to measure the online conversion impact of integrating and leveraging product-specific word of mouth in existing marketing programs and contexts. Using reporting and analysis tools provided by each client, Bazaarvoice found that recipients of word of mouth marketing messages converted at higher than average rates and in two of three cases, spent considerably more per online order or unique visitor than non-recipients. In all cases, the retailers integrated customer word of mouth into existing marketing programs that offered higher reach and impact than more “manual” forms of marketing message distribution, such as an individual consumer emailing a review for a favorite product to a relatively small group of friends or discussing a review at the water cooler. Allowing a customer review to piggyback on existing marketing programs exponentially increases the audience size for that review, thereby increasing its marketing impact and ROI. For example, a customer rating and review that is shared with millions of consumers via a retailer’s monthly email newsletter has a much greater audience than a rating and review that simply “lives” on a product web page, usually buried deep with the multi-level structure of a large e-commerce web site.

Case Study: CompUSA Finds ROI in Search Engine Indexing of Reviews

CompUSA launched customer ratings and reviews on CompUSA.com just in time for the 2005 holiday shopping season. Within weeks, leading search engines indexed the web pages that contained CompUSA’s rapidly growing number of customer reviews.

Although the algorithms that directly control the search engine rankings of web sites and specific pages within those sites remain a mystery even to many search marketing experts and industry insiders, consumer-generated media is believed to be search engine friendly for several reasons. First, it is authentic in both structure and tone, making it a more credible form of information in the eyes (or algorithms!) of many search engines. Second, it is often highly linked both to and from other web sites, elevating its importance and relevance. Last, consumer-generated media changes frequently, and each new post, comment, or thread scores points for freshness. The release of blog-specific search capabilities by Google, Yahoo, and other web search vendors is evidence of the importance of consumer-generated media to the online advertising and search marketing industry.

By late November, roughly six weeks after CompUSA’s launch of ratings and reviews, thousands of new visitors were arriving at CompUSA.com each week as a result of reviews-related web searches. These consumers were not responding to CompUSA’s paid search advertisements, which are displayed on search engine results pages for a range of brand-, category-, and product-specific search phrases and keywords numbering in the thousands. Rather, these consumers were finding links to CompUSA’s customer ratings and reviews – and therefore products – embedded within the natural results for search phrases such as “XBOX 360 reviews” as a result of search engine indexing – a high velocity, high reach “distribution” activity – and their own explicit search queries.

Actual compUSA.com page in Google Search Results

Figure 1. Actual compUSA.com page in Google Search Results

Bazaarvoice worked with CompUSA to measure and compare the behavior of site visitors acquired by reviews-related searches to the average CompUSA.com visitor. The companies found that the test group outperformed the control group in virtually every measure. Specifically, they found that these visitors:

In addition, the companies conducted a second analysis for an eight-week period and found that indexed customer ratings and reviews, when viewed as a visitor acquisition source, delivered greater sales per unique visitor than all four major search engines (AOL, Yahoo, MSN and Google) on an aggregate basis, although these same search engines were the primary vehicles used by consumers to find CompUSA’s customer ratings and reviews!

As a result of these findings, CompUSA has launched a series of promotional contests and online incentives designed to aggressively drive the growth of customer review submissions on CompUSA.com, thereby increasing the volume of reviews available for indexing on the major search engines. In this case, the incredible distribution velocity and reach of leading search engines created a word of mouth success story for CompUSA, an established multi-channel retailer.

Case Study: PETCO Uses Aggregated Ratings to Drive Conversions and Shopping Convenience

Like CompUSA, PETCO deployed customer ratings and reviews on PETCO.com in mid-October 2005. Almost immediately, the retailer began heavily promoting the feature on its home page, order confirmation page, order and shipping confirmation emails, and through the use of low-cost contests, such as a $100 gift card giveaway.

By mid-December, PETCO had accumulated a large enough volume of ratings and reviews to begin using aggregated ratings as a means of segmenting its online product catalog, which consists of over ten thousand unique products. For example, products with multiple reviews and a high average overall rating (the threshold for “high” changes over time as products accumulate more ratings and reviews) were categorized as “Top Rated Products.” Segmenting the product catalog in this manner allowed PETCO.com to offer a “Top Rated Products” category within each of its seven key departments. As a result, visitors to PETCO.com are able to navigate the site by selecting a department, such as “Dog”, and then selecting “Top Rated Products” in addition to any of their traditional Dog department categories, such as “Biscuits & Treats.” The Top Rated Products navigation option acts as a powerful distribution mechanism for customer ratings and reviews, as it elevates the visibility and potential impact of each customer rating and review unit related to a highly rated product.

PETCO.com Top Rated Products Sub-Category

Figure 2 – PETCO.com Top Rated Products Sub-Category

Using PETCO’s web analytics system, Bazaarvoice and PETCO began analyzing the conversion behavior of PETCO.com visitors that chose the Top Rated Products navigation path within any pet department. The companies found that this visitor group had a higher overall conversion rate than other browsers (visitors that actively navigate the site to look at products). More recently, Bazaarvoice repeated this same analysis for a twelve week period and found that compared to average browser performance, visitors that utilized Top Rated Products to browse PETCO.com converted at considerably higher rates on both a same-session and multi-session basis (10% and 49%, respectively).

In addition, the second analysis evaluated PETCO’s use of average customer rating as an on-site search results refinement option. When PETCO.com visitors use the site’s search function to search for “dog treats,” for example, they can filter search results to just include the most highly rated products. In the second analysis, Bazaarvoice found that the sort by rating refinement option was the top performing sort method. Compared to the second best method, sort by rating delivered higher sales per unique visitor on both a same-session and multi-session basis (22% and 41%, respectively).

PETCO.com Search Results-Sort by Customer Rating

Figure 3: PETCO.com Search Results-Sort by Customer Rating

Much like CompUSA, PETCO continues to invest in the development of its reviewer community and in creative uses of customer ratings and reviews content. The retailer has leveraged ratings and excerpts from actual customer reviews in email newsletters and in on-site advertising creative with great success. Most significantly, PETCO knows that its own web site and common functionality like site search can be leveraged to increase the reach and impact of customer ratings and reviews, resulting in greater ROI from their investment in word of mouth marketing.

Case Study: Burpee Boosts RSS Feed Response Rates Using Featured Product Reviews

Burpee’s launch of customer ratings and reviews in early 2006 was the company’s second investment in “social media” and its enabling technologies. Several months earlier, the retailer decided to re-purpose its email newsletter content to provide an RSS feed to is more tech savvy customers. The Burpee.com home page links to a web page that contains detailed information about how to subscribe to the RSS feed and information on RSS in general, including links to RSS readers and other resources. Thanks to Burpee’s active promotion of RSS and near daily updates, their feed’s subscriber base continues to grow each month.

In early February 2006, just weeks after launching customer ratings and reviews, Burpee decided to incorporate selected customer reviews alongside products featured in the RSS feed. Previously, the “Featured Product” edition of the feed simply included a product image, short description, and links to the product page on Burpee.com. Incorporating just a single customer review provided subscribers with more detailed information about the product as well as an authentic customer opinion or personal account of using the product. After seven weeks, Bazaarvoice and Burpee began to analyze daily clickthroughs from the RSS feed. During this same period, average daily clickthroughs steadily grew, but the companies found that days that included a customer review had a 43% higher number of clickthroughs, on average, than days that did not include customer reviews. For Burpee, this finding provided clear proof that online word of mouth could drive its business growth in much the same way that offline word of mouth fueled the success of its original catalog business.

Burpee RSS Feed with Featured Product Review

Figure 4: Burpee RSS Feed with Featured Product Review

Like PETCO, Burpee leveraged an existing marketing asset to increase the exposure of customer ratings and reviews of their products. While Burpee’s RSS feed, which is inherently permission-based, does not yet offer the reach or velocity advantages of a Google web search, it provided the perfect opportunity for measuring the impact of customer word of mouth on an existing marketing program. Since conducting the analysis above, Burpee has gone one step further by building out a company blog that features customer reviews in many of its regular posts.

About the Authors

Brant Barton is the co-founder and VP of Business Development of Bazaarvoice. He previously served as Bazaarvoice’s VP of Client Services and has worked directly with Bazaarvoice clients, including CompUSA, PETCO, Burpee, and others, to integrate customer ratings and reviews into their web sites and online marketing programs. Prior to Bazaarvoice, Brant was Director of Account Services at Coremetrics, where he advised leading retail and e-commerce brands on how to improve online conversion using applied web analytics. He has presented on customer ratings and reviews at a number of industry conferences, including Shop.org’s 2006 Online Marketing Workshop, the ad:tech 1MPACT conference series, and SXSW Interactive 2006.

About Bazaarvoice

Based in Austin, Texas, Bazaarvoice offers outsourced technology, community management services, analytics, and syndication to encourage and harness word of mouth marketing, and bring it closer to a company’s brand and customer experience. The company’s flagship hosted and fully managed customer ratings and review service allows businesses to enable, encourage, and analyze customer ratings and reviews on their website. With Bazaarvoice, companies can empower their customers to share honest opinions and influence each other to make more informed and rewarding purchase decisions. Clients like CompUSA, Overstock.com, and PETCO benefit from a credible and reliable customer-to-customer community, without having to delve into complex IT work or the laborious process of community management. Bazaarvoice is a winner of the “Red Herring 100 North America” award by Red Herring magazine. For more information, please visit the company’s website at www.bazaarvoice.com or email [email protected].

Publication History

This whitepaper was originally prepared for the member community of the Word of Mouth Marketing Association (WOMMA). A version of it will be included in WOMMA’s publication, “Measuring Word of Mouth, Volume 2.”