September 15, 2017

Yelp reviews don't capture care quality, study finds—but here's why they still matter

Researchers found little-to-no correlation between online consumer ratings of physicians and quality measures, suggesting that these online sites should not be exclusively used to evaluate a doctor's quality of care, according to a new study published in the Journal of the American Medical Informatics Association.

Get the resources to successfully attract and onboard new physicians

Study details

For the study, researchers at Cedars-Sinai Medical Center assessed consumer reviews of 78 doctors across eight specialties on five different online review sites: Healthgrades, Yelp, Vitals, RateMDs, and UCompareHealthCare. The researchers then compared the ratings with quality measures, such as length of stay, adherence to practice guidelines, readmission rates, and peer reviews from physicians and administrators.

5 myths physicians believe about patient experience

The researchers found virtually no correlation between the online reviews and providers' performance on quality measures. For instance, among the doctors who were in the bottom 25 percent of their specialty-specific performance scores, only 5 to 32 percent had ratings on the online review sites that were also in the bottom 25 percent.

The researchers did find, however, a strong correlation across consumer rating sites, as the score of one doctor on one site was very predictive of his or her score on another rating site.

Discussion

According to Timothy Daskivich, the lead author of the study and the director of health services research in Cedars-Sinai's surgery department, this lack of correlation is likely due to the fact that patients are unable to accurately assess the quality of the clinical aspects of their care. "In the health care setting, unless you have very poor outcomes, it's hard (for patients) to discern whether or not the quality of care that is being provided is good or bad," he said.

The researchers recommended further studies into exactly what consumer review sites aim to assess. For instance, they pointed out that if consumer review sites clarified that they provide feedback on patient experience—not quality of care—it might help consumers understand how to best use the information.

Brennan Spiegel, a gastroenterologist and one of the co-authors of the study, said, "It may be that these ratings are a good measure of the front-office service or the interpersonal style of the physician." He added, "We're not saying that there's no value to these online ratings—we're saying don't confuse those ratings in any way, shape, or form with the actual technical skill."

To get a more comprehensive recommendation on a provider, Daskivich suggested that patients—in addition to considering consumer reviews—also seek input from their PCP or use the CMS' Physician Compare website, which includes quality rankings.

Register for the National Meeting

2017-18 Physician Executive Council national meeting

Response from the consumer review sites

According to Modern Healthcare, some of the review websites assessed in the study agreed that they provide insight into patient experience, not clinical quality. "People don't make a decision based on patient ratings," Andrea Pearson—CMO for Healthgrades, which also features quality data on the hospitals physicians are affiliated with—said. "They make it based on patient ratings, experience, and quality outcomes."

However, a spokesperson for Yelp, was more critical of the research, saying that it looked at an "extremely small [sample] size" of doctors and focused on factors that the public couldn't easily access. "We wouldn't necessarily expect those internal metrics to correspond with Yelp reviews since they're measuring different things," the spokesperson said.

Daskivich acknowledged the study's small sample size. However, he said the results would likely hold up in a larger study because the findings were consistent across specialties and the data on quality measures was particularly robust (Tracer, Bloomberg, 9/8; Castellucci, Modern Healthcare, 9/11; Vaidya, Becker's Infection Control & Clinical Quality, 9/11).