Can your supervisor be trusted to review your work?

Rating the performance of employees can be one of a supervisor’s least favorite tasks. Often times supervisors give better reviews than employees deserve because they want to be liked, play favorites or worry that unfavorable reviews will reflect poorly on their own performance.

The reviews are even more complicated when the criteria are subjective — does a person work well with a team or think through complex issues? — rather than objective, like how many widgets does the employee produce each week.

Companies have attacked the age-old problem by establishing committees that review the ratings that supervisors give employees and adjust to remove some of the bias. But researchers have struggled to analyze how well the committees do their job. Companies tend to be reluctant to share the personnel data.

“Because it is so common in the industry, I think it’s one of those things where a lot of people consider it to be a good practice so it just spreads from one company to another,” said University of Missouri accounting professor Will Demeré.

Mr. Demeré and his colleagues, Karen Sedatole of Emory University and Alexander Woods of the College of William and Mary, finally convinced a company to share three years’ worth of data on performance reviews done with the help of the committees. They agreed not to identify their subject, which is described only as a “large multinational organization” that employs auditors.

What the researchers found is that the committees are more likely to believe employees didn’t perform as well as their supervisors think they did and less likely to believe that employees weren’t as bad as supervisors said they were.

“They would more often adjust high ratings down than adjust low ratings up,” Mr. Demeré said.

The researchers also determined that the committees tended to cluster ratings in the middle.

That makes it harder to identify workers who should be singled out for rewards and promotions as well as poor performers who might need additional training or help, Mr. Demeré said.

He said law, accounting and other professional services firms have been using the panels — known as calibration committees — for more than 20 years. A worker’s performance is harder to measure in those types of firms than it is in a manufacturing company, where performance can be judged on output and more objective measures.

The committees are more common in organizations where many employees do the same work, but are judged by multiple supervisors — each with his or her own idea about what constitutes a good job. The committees tend to be staffed by higher-level managers than the supervisors, although supervisors are sometimes included, Mr. Demeré said.

He said the data the researchers looked at indicates that supervisors tended to take the committees’ adjustments to their reviews into account when evaluating the same employee the following year.

Mr. Demeré said supervisors adjusted ratings higher to the same extent that the committee adjusted them higher the previous year. However, if the committee lowered an employee’s evaluation, the supervisor did not lower it to the same extent the following year.