Michael Luca

Max H. Bazerman

How organizations—including Google, StubHub, Airbnb, and Facebook—learn from experiments in a data-driven world. Have you logged into Facebook recently? Searched for something on Google? Chosen a movie on Netflix? If so, you've probably been an unwitting participant in a variety of experiments—also known as randomized controlled trials—designed to test the impact of different online experiences. Once an esoteric tool for academic research, the randomized controlled trial has gone mainstream. No tech company worth its salt (or its share price) would dare make major changes to its platform without first running experiments to understand how they would influence user behavior. In this book, Michael Luca and Max Bazerman explain the importance of experiments for decision making in a data-driven world. Luca and Bazerman describe the central role experiments play in the tech sector, drawing lessons and best practices from the experiences of such companies as StubHub, Alibaba, and Uber. Successful experiments can save companies money—eBay, for example, discovered how to cut $50 million from its yearly advertising budget—or bring to light something previously ignored, as when Airbnb was forced to confront rampant discrimination by its hosts. Moving beyond tech, Luca and Bazerman consider experimenting for the social good—different ways that govenments are using experiments to influence or “nudge” behavior ranging from voter apathy to school absenteeism. Experiments, they argue, are part of any leader's toolkit. With this book, readers can become part of “the experimental revolution.”

Decision Making in a Data-Driven World

The Power of Experiments

Power of Experiments

Harvard Business Review

Andrew Ng

Beena Ammanath

Bhaskar Ghosh

A year of HBR's essential thinking on tech—all in one place. Easy-to-use AI tools, contactless commerce, crypto for business, the mature metaverse—new technologies like these are reshaping organizations at the hybrid office, on factory floors, and in the C-suite. What should you and your company be doing now to take advantage of the new opportunities these technologies are creating—and avoid falling victim to disruption? The Year in Tech 2023: The Insights You Need from Harvard Business Review will help you understand what the latest and most important tech innovations mean for your organization and how you can use them to compete and win in today's turbulent business environment. Business is changing. Will you adapt or be left behind? Get up to speed and deepen your understanding of the topics that are shaping your company's future with the Insights You Need from Harvard Business Review series. Featuring HBR's smartest thinking on fast-moving issues—blockchain, cybersecurity, AI, and more—each book provides the foundational introduction and practical case studies your organization needs to compete today and collects the best research, interviews, and analysis to get it ready for tomorrow. You can't afford to ignore how these issues will transform the landscape of business and society. The Insights You Need series will help you grasp these critical ideas—and prepare you and your company for the future.

The Year in Tech, 2023: The Insights You Need from Harvard Business Review

Edward Ludwig Glaeser

Hyunjin Kim

We demonstrate that data from digital platforms such as Yelp have the potential to improve our understanding of gentrification, both by providing data in close to real time (i.e. nowcasting and forecasting) and by providing additional context about how the local economy is changing. Combining Yelp and Census data, we find that gentrification, as measured by changes in the educational, age, and racial composition within a ZIP code, is strongly associated with increases in the numbers of grocery stores, cafes, restaurants, and bars, with little evidence of crowd-out of other categories of businesses. We also find that changes in the local business landscape is a leading indicator of housing price changes, and that the entry of Starbucks (and coffee shops more generally) into a neighborhood predicts gentrification. Each additional Starbucks that enters a zip code is associated with a 0.5% increase in housing prices.

Using Yelp Data to Quantify Neighborhood Change

Measuring Gentrification

Georgios Zervas

Consumer reviews are now a part of everyday decision-making. Yet, the credibility of reviews is fundamentally undermined when business-owners commit review fraud, either by leaving positive reviews for themselves or negative reviews for their competitors. In this paper, we investigate the extent and patterns of review fraud on Yelp, a popular consumer review platform. Because one cannot directly observe which reviews are fake, we focus on reviews that Yelp's algorithmic indicator has identified as fraudulent. Using this proxy, we present four main findings. First, roughly 16% of restaurant reviews on Yelp are identified as fraudulent, and tend to be more extreme (favorable or unfavorable) than other reviews. Second, a restaurant is more likely to commit review fraud when its reputation is weak, i.e., when it has few reviews, or it has recently received bad reviews. Third, chain restaurants - which benefit less from Yelp - are also less likely to commit review fraud. Fourth, when restaurants face increased competition, they become more likely to receive unfavorable fake reviews. To reinforce our interpretation that fake reviews are driven by economic incentives, we then collect a second data set of businesses that were caught soliciting fake reviews through a sting conducted by Yelp. Results from these data are consistent with our main results, further identifying economic incentives behind a business's decision to leave fake reviews. Taken in aggregate, these findings highlight the extent of review fraud and suggest that a business's decision to commit review fraud responds to competition and reputation incentives.

Reputation, Competition, and Yelp Review Fraud

Fake it Till You Make it

Data from digital platforms have the potential to improve our understanding of gentrification and enable new measures of how neighborhoods change in close to real time. Combining data on businesses from Yelp with data on gentrification from the Census, Federal Housing Finance Agency, and Streetscore (an algorithm using Google Streetview), we find that gentrifying neighborhoods tend to have growing numbers of local groceries, cafes, restaurants, and bars, with little evidence of crowd-out of other types of businesses. For example, the entry of a new coffee shop into a zip code in a given year is associated with a 0.5 percent increase in housing prices. Moreover, Yelp measures of local business activity provide leading indicators for housing price changes and help to forecast which neighborhoods are gentrifying.

Nowcasting Gentrification

National Bureau of Economic Research

Harvard Business School

Jungmin Lee

Weijia Dai

Ginger Zhe Jin

Consumer review websites such as Yelp.com leverage the wisdom of the crowd, with each product being reviewed many times (some with more than 1,000 reviews). Because of this, the way in which information is aggregated is a central decision faced by consumer review websites. Given a set of reviews, what is the optimal way to construct an average rating? We offer a structural approach to answering this question, allowing for (1) reviewers to vary in stringency (some reviewers tend to leave worse reviews on average) and accuracy (some reviewers are more erratic than others), (2) reviewers to be influenced by existing reviews, and (3) product quality to change over time. We apply this approach to reviews from Yelp.com to derive optimal ratings for each restaurant (in contrast with the arithmetic average displayed by Yelp). Because we have the history of reviews for each restaurant and many reviews left by each reviewer, we are able to identify these factors using variation in ratings within and across reviewers and restaurants. Using our estimated parameters, we construct optimal ratings for all restaurants on Yelp and compare them to the arithmetic averages displayed by Yelp. As of the end of our sample, a conservative finding is that roughly 25%-27% of restaurants are more than 0.15 stars away from the optimal rating, and 8%-10% of restaurants are more than 0.25 stars away from the optimal rating. This suggests that large gains could be made by implementing optimal ratings. Much of the gains come from our method responding more quickly to changes in a restaurant's quality. Our algorithm can be flexibly applied to many different review settings.

An Application to Yelp.com

Optimal Aggregation of Consumer Ratings

Do online consumer reviews affect restaurant demand? I investigate this question using a novel dataset combining reviews from the website Yelp.com and restaurant data from the Washington State Department of Revenue. Because Yelp prominently displays a restaurant's rounded average rating, I can identify the causal impact of Yelp ratings on demand with a regression discontinuity framework that exploits Yelp's rounding thresholds. I present three findings about the impact of consumer reviews on the restaurant industry: (1) a one-star increase in Yelp rating leads to a 5% to 9% increase in revenue, (2) this effect is driven by independent restaurants; ratings do not affect restaurants with chain affiliation, and (3) chain restaurants have declined in market share as Yelp penetration has increased. This suggests that online consumer reviews substitute for more traditional forms of reputation. I then test whether consumers use these reviews in a way that is consistent with standard learning models. I present two additional findings: (4) consumers do not use all available information and are more responsive to quality changes that are more visible and (5) consumers respond more strongly when a rating contains more information. Consumer response to a restaurant's average rating is affected by the number of reviews and whether the reviewers are certified as "elite" by Yelp, but is unaffected by the size of the reviewers' Yelp friends network.

Michael Luca

About

Books

Join the club