Jeffrey S. McCullough, Ph.D
Division of Health Policy and Management
Increasingly, patients use consumer-generated online ratings as a source of physician quality information. There are substantial concerns about whether websites provide useful and unbiased information. Critics worry that online ratings may unfairly blemish physician reputations and mislead patients. Conversely, advocates argue that these ratings have the potential to enlighten consumers and improve care quality. Despite rising consumer interest, little is known about the actual association between online rating and physician quality.
We addressed these issues in a series of three papers. The first studied the diffusion of consumer-generated online ratings for physicians (Gao et al., 2012). The second study matched online ratings to traditional measures of patient satisfaction (Gao et al., forthcoming). Finally, the third study examined the relationship between online ratings and physician quality (Grey et al., 2015).
Online ratings growth
We found that consumer-generated online ratings are widespread and growing rapidly. The average number of ratings per physician were modest; however, this was driven by the rapid growth of these ratings. The number of ratings per physician grew quickly over time – but this is overwhelmed in the short run by the number of newly rated physicians who have only one consumer rating.
Physician ratings typically high
We found no evidence that online ratings were driven by disgruntled patients. Online consumer ratings were quite high on average. Physicians with relatively high conventional patient satisfaction scores were more likely to receive online ratings. Online ratings were, if anything, higher than conventional/offline ratings for the same physicians. This suggests that being rated online is a signal of patient satisfaction. The predominant challenge with using these ratings as a patient satisfaction measure is that they are uniformly positive; consequently, it’s difficult to detect differences across physicians.
Ratings and clinical quality
Ultimately, it is important for consumers to understand whether online ratings reflect differences in clinical quality. We partnered with the American Board of Internal Medicine (ABIM) to study the relationship between online ratings and diabetes quality. We extracted information from eight leading health-based websites that were publicly available and free of charge. These data were matched to a variety of quality and patient satisfaction measures for ABIM members.
We found no evidence that website ratings were associated with clinical quality. Associations between website ratings and traditional, offline, patient satisfaction measures were statistically significant. These studies suggest that consumer generated online ratings may serve as an important source of consumer satisfaction information, but consumers should be cautious when using online ratings.
References and links
Gao GG, McCullough JS, Agarwal R, Jha AK. A Changing Landscape of Physician Quality Reporting: Analysis of Patients’ Online Ratings of Their Physicians Over a 5-Year Period. 2012;14(1).http://www.jmir.org/2012/1/e38/
Gao GG, Greenwood BN, McCullough JS, and Agarwal R. Vocal Minority and Silent Majority: How do Online Ratings Reflect Population Perceptions of Quality? MIS Quarterly; Forthcoming.
Gray BM, Vandergrift JL, Gao GG, McCullough JS, and Lipner RS. Website Ratings of Physicians and Their Quality of Care. JAMA Intern Med. 2015;175(2):291-293.http://archinte.jamanetwork.com/article.aspx?articleid=1936577
Jeffrey McCullough teaches Health Finance II to full-time and executive MHA students at the University of Minnesota. His research centers on health care finance and economics, with a focus on the role of technology and innovation in health care.
Posted by Mona Rath