Top | Oyster Food and Culture

Wine judges, do we trust them?

(post, LouAnn Con)


primary-image, l

... and if we do, should we?  When I select my wine, the scores posted along with the description of the wine certainly influence my decision.  My assumption is that the educated folks making the decisions are professionals after all, and as for me, I'm imbibing purely for pleasure.  Sure, I know what I like, but I am not confident enough to voice my opinion when pushed for details.  In truth, I wilt like yesterday's parsley if questioned by someone I perceive as more knowledgeable.  But should I be so hard on my abilities?  I've consumed wine for a number of years, and really enjoy the experience.  My husband, while he drinks, is content with a glass of wine at dinner. 

If I had my way, and for quest of scientific knowledge only,  I would open a few bottles of zin to do comparison tasting.  But being a properly raised mid-western gal I have waste issues, as I know my husband would not keep up his end.  (My dream job belongs to Jeffery Steingarten at Vogue)   Then there is the matter that we get up at 5 am to run.  If you've never tried it, I do not recommend early rising after a delightful evening of sipping the juice.

Back to my original question, a study was performed at the California Wine Fair that showed only 30 of the 65 judges achieved consistent results.  Each judge received a flight of 30 wines, in which were embedded triplicate samples poured from the same bottle.  Only 10% of the judges replicated their selections in a single award category.  Put another way, 90% of the judges could not repeat their work!  90% - that's huge!  To get a better picture, 80% of those judges scored the same wine from as a gold medal winner to no medal.

Given that ratings can directly affect the price charged for a wine I wonder why this issue is not raised, or the vineyards challenge the rankings.  Before you say, ah but its just the California judges that have this problem, I have news for you.  The American Association of Wine Economists AAWE agrees that this problem is systemic across the industry.   (Bet you didn't know there was an American Association of Wine Economists)

Given the issue, and implications, the industry is trying out various solutions.  One suggestion for improving the rating system was to take the average of a small group of judges.  When this method was used, the results proved more consistent.  I do not know about you, but my self confidence soared after reading this news.  Now, I know that the experts face the same issues, they can be overwhelmed as well.