Rating Wines Controversy: It's ba-a-a-a-a-ck!

There's an interesting series of articles, responses, comments, and even a "Point-Counterpoint" column over on Palate Press, a really fine internet-based wine magazine, regarding a poll on rating wines...the old points controversy.

What I propose is---before going to the Palate Press page and reading up---we do our own voting in our own poll here. Same poll as Palate Press, but without any influence of what is being said on Palate Press.

Then, once you've voted, go on over there and read the poll (and vote there if you're so inclined) and read the fascinating comments and reactions and stances taken.

It's a controversy that will never go away, of course; but it is still interesting to hear what people say about both sides.

The Palate Press address is http://www.palatepress.com

This is the topic that will not die! Thanks for the link, the discussion was actually intelligent. You'll have to excuse my surprise but in general I find must discussions like this on the web to be, well not so. I do agree with the point that the scoring system does seem to bring in air of objectivity to a subject that is by its very nature not objective. It do find it interesting the amount of wine that seems to be crouched around the mythic 90 point zone. Related to that I can't tell you the number of times I've been disappointed in myself because I really, really loved an 88 pointer.

BTW can someone with proper historical understanding explain to me how a 100 point scale seems to in reality only use the numbers 70, 75, 88, and 90-97.

Kelly
Yes, that faux objectiveness that some assume iro points is perhaps the nub of the issue. Indeed the phrase '98 pointer' (whatever the number) is perhaps the most crass usage of all.

In reality I've no great objection to a score, which can be useful if the TN balanced positives and negatives, but the tasting note is much more important. The points on their own are like drooling over a wine bottle, after pouring the contents down the sink.

and if I had to choose a scale, at least 1-5 stars uses all the scale, plus doesn't claim false 'precision', so I'd use that if I were ever to get into scoring wine.

regards

Ian

Kelly, there never was any such thing as a "100 point scale". Even when people inaugurate what they call a 100 point scale, they automatically set a bottom cut off of whatever they individually feel comfortable with...so that the scale automatically becomes a 50 point, 30 point, 25 point, 10 point...whatever.

It's the appearance of objectivity only, while remaining totally and entirely subjective, of course. And of course all this is that curious human phenomenon of the "disappearing scale"---if there's one guy looking at a (real) 50 point scale and considering only the half of wines he sees that are over his imaginary personal line of acceptance for consideration, then the next guy reaches a new height when he announces he's only looking at the top 25% of wines that are worthy of his notice. Then you get the people who don't even bother looking at anything under 90 points, trying to find the most exclusive wines out there.

Of course, once you've reached that point, it's actually not about wine anymore. It's about the scores, or the exclusivity, or what your possessions say about you, than it would ever be about wine.

I'll leave it up to those who study and practice the science and art of statistics and psychology to explain in detail why all this is so.

Funny side story: Last weekend I attended my grand daughter's (she's about three years old) soccer league game. It was pretty hilarious, and at the same time frustrating, for me to watch theses kids running around on the grassy field. It went back and forth, back and forth, for the required time. The kids mostly had fun; the parents sat in chairs on the sideline and mostly had fun too. All very social.

At the end of the "game" I turned to my daughter, the Mom of the grandaughter, and said, "What's the final score?" She grinned at me and said, "Oh, nobody keeps score at these things. Only the Dads; because it's only the Dads that care about that stuff."

I stopped "keeping score" with wines a long, long time ago. The wines don't seem to be any better or worse for it, either.

The problem, as noted, is the false sense of precision of the 100 point scale. But I'm actually ok with someone using 100 point scale (go Gilman! onward Meadows!), 20 point scale (alright Jancis!), 5 star , letter grades, no ratings, "I would buy this," etc. It basically comes down to the READER understanding that these are just one person's (well, sometimes 1 panel's) opinion, probably of a bottle, probably at a point in time.

Opening the latest VFTC, if I read a glowing note on the '09 Brun "L'Ancien" and a glowing note on the 09 Vissoux "Traditionelle" , the points tell me that Gilman loved both, but the Vissoux just a touch more (93 v. 92). It is a piece of information, I pay more attention to notes, but it is one more piece.

Ratings aren't the problem, it's the reader (and in some cases the critic) who regard them as somehow objective.

As is known, when I publish wine reviews they are accompanied by a score and I use the 100 point system. Despite that I voted that I prefer wine reviews without scores.

My logic is quite simple...when I first started using the 100 point system (indeed a 50 point system) those scores were entirely a reference for my personal benefit and were not published. As a perhaps interesting side note my use of this scoring system pre-dates Parker by more than 20 years but, as he published scores long before me, I am happy for him to take the credit. As I have said before, I only started publishing scores when readers started demanding them and editors who take readers quite seriously insisted on them.

I continue to make no apology for scores, for although they may give the illusion of full control and objectivity, all who have been "into" wine for more than a short while realize that a score is nothing more than that particular critic's summation of the quality of a given wine. More important - as human beings we tend to put nearly everything on a scale. We listen to a concert and we evalue the performance as outstanding, excellent, very good, good, mediocre, poor, bad, terrible; we see a person of the opposite sex and we decide whether he/she is physically gorgeous, very attractive, attractive, not very attractive, unattractive or ugly; and on and on to ever field of human thought and evaluation. We probably do not formalize those "ratings" but they are there and if called on to give them a numerical score from 1-5, 1-10, 1-20, 1-100 we could do so.

I do concur that for those who may not be fully in the know, it behooves all critics who use scores to perdiocally remind their readers that scores are nothing more than two or perhaps three digits at the end of a review. More...that while the score may sum up that critic's quality judgement, it says nothing whatever about whether the reader will gain pleasure from the wine under review. Directions for whether one will or will not enjoy a wine are not in the score - only in the tasting note.

I suspect that one day the debate over cork versus screwcaps will fade into oblivion. The debate over scores will go on until I and most of my colleagues will be long in a box. Whether that box would be measured as attractive, as "comfortable" and as "well equipped" to take us on our voyage to wherever will be left for some pundit qua critic to decide.

Best
Rogov

I find tasting notes so much more helpful than point ratings, because although both are subjective to personal taste and opinion, at least some detail as to why someone liked a wine is more helpful to me than an arbitrary number assigned (really - what IS the difference between 91 and 92 points?!?!).

I also have a major beef with Cellar Tracker on this issue (although I love them for everything else about cellar organizing). The problem is that any idiot can assign a 95 point rating on an absolutely crap wine and conversly people may underappreciate a really great wine that they don't get, or isn't jammy enough for them. Waaaaaay to oversimplified for my taste. I'd much rather understand your palate and preference for Old vs. New World wines and take that into consideration when deciding if I will take your recommendation than just going by a number. One person's 70 is another's 90.

Daniel:

Let's say you review three wines in succession and apply scores to those three wines.

Let's say the three wines in question are

a Slovenian "orange" wine made from Friulano in open-fermentation vats and aged in terracotta amphorae for 6 years.

a Santa Maria Valley Pinot Noir made from late-picked, extra-ripe and partially raisinated grapes and aged for 24 months in a regimen of "200% new French Oak with heavy toast.

a Beaujolais Nouveau.

Let's further say that each wine receives 90 points from you---and when I get the review, unfortunately all I get is the point scores (because that's all my friend who sent it cares about so he edited out all those words ).

What could that possibly mean to me? Is a 90 point Orange Wine from Slovenia "equal to" a Beaujoais Nouveau?

Is a heavily oaked, high sugar/high alcohol new world pinot noir equivalent to a one day old semi-carbonically macerated gamay?

(And before you answer, I think I know what it would mean TO YOU. What I'd like to know is what you think that might mean to A READER as well----i.e, what you intended, versus what would be perceived.)

The problem is that any idiot can assign a 95 point rating on an absolutely crap wine and conversly people may underappreciate a really great wine that they don't get, or isn't jammy enough for them.

Well, yeah, that's always been the case...but that's not specific to points, Joy. Points do make that tendency easier though. And even though I don't necessarily agree with what Blake Gray says in his Point-Counterpoint comments on Palate Press, I will agree that it is always....always the responsibility of the buyer/reader to sift out the idiots from the perceptive and knowledgeable self-appointed givers of value.

Hoke, Hi...

You talk about a friend who sends you nothing but the score and as what that should mean to you. If I may be so bold, it means that your friend is a schmuck. After all, what he has done is the equivalent of having sent you the hub cap or hood ornament of a Lamborghini and asked you to use that in making your judgement about the car.

Best
Rogov

Hoke, Hi...

You talk about a friend who sends you nothing but the score and as what that should mean to you. If I may be so bold, it means that your friend is a schmuck. After all, what he has done is the equivalent of having sent you the hub cap or hood ornament of a Lamborghini and asked you to use that in making your judgement about the car.

Best
Rogov

Thus proving my belief that, like a Lamborghini hub cap or hood ornament in the absence of anything else, a point score is meaningless. Objectively speaking.

However, if one happens to possess a Lamborghini, and is in need of a replacement hub cap or hood ornament, then objectively speaking, either one of those objects becomes a thing of immediate and immense value. Einstein was right: everything is relative.

If I send you a Lamborghini hub cap or hood ornament, do not think I am a schmuck----consider me an optimist and admirer of yours, one who sends you a "starter kit" for the Lamborghini that you would so love to possess.

I find tasting notes so much more helpful than point ratings, because although both are subjective to personal taste and opinion, at least some detail as to why someone liked a wine is more helpful to me than an arbitrary number assigned (really - what IS the difference between 91 and 92 points?!?!).

Joy, Hi...

If the truth be told there is no statistical significance between 91 and 92 points. On the other hand, there is a major difference between 89 and 90 points, the one point in this case raising the wine from the category of very good to that of excellent.

From a somewhat varied point of view, let's say you were working towards your PhD at university and in one of your courses received instead of a B a B- . Considering that two or three B- grades would have you dropped from the program, that little minus sign may have great significance.

The problem is that any idiot can assign a 95 point rating on an absolutely crap wine and conversly people may underappreciate a really great wine that they don't get, or isn't jammy enough for them. Waaaaaay to oversimplified for my taste. I'd much rather understand your palate and preference for Old vs. New World wines and take that into consideration when deciding if I will take your recommendation than just going by a number. One person's 70 is another's 90.

When talking about Cellar Tracker or other programs we should be aware that the vast number of tasting notes are posted not by professionals but by amateurs. Indeed some of those amateurs are as well equipped as the most highly valued of professional critics, but a not insignificant number have the wine sense of a mentally retarded mosquito. It falls on the reader of those notes to understand and be discriminating in just whose tasting notes and/or scores one reads.

With re professionals (and I agree that not all who write criticism and get paid for it are true professionals), I believe you will find major correlation between the scores of many of the better critics. That is not collusion on their part. It is that they have individually and collectively built up a repertoire of standards and those can be reflected in both tasting notes and scores. For sure there is sometimes wide variation but that is more the exception than the rule and in those cases one follows that critic that most often speaks to his/her personal taste.

Best
Rogov

Thus proving my belief that, like a Lamborghini hub cap or hood ornament in the absence of anything else, a point score is meaningless. Objectively speaking.

We have never disagreed that a score on its own is meaningless. You will find no-one who agrees with you more strongly on that.

If I send you a Lamborghini hub cap or hood ornament, do not think I am a schmuck----consider me an optimist and admirer of yours, one who sends you a "starter kit" for the Lamborghini that you would so love to possess.

Considering that a set of four hub caps for the 1990 Lamborghini Countach after which I so devotedly lust costs about US$ 1050, I would consider you a very generous admirer.

Oh yes, let us also keep in mind that in German the word schmuck means "jewel".

Best
Rogov

Oh yes, let us also keep in mind that in German the word schmuck means "jewel".

Wow! I had no idea what those people were calling me. It didn't seem like they were saying "jewel" at the time, though.

I think on the 100 point scale I'm going to get 177 points of vin into my face tonight. The scale is additive isn't it?

I think on the 100 point scale I'm going to get 177 points of vin into my face tonight. The scale is additive isn't it?

All the wineries that use them would like to think they are cumulative.

If the truth be told there is no statistical significance between 91 and 92 points. On the other hand, there is a major difference between 89 and 90 points, the one point in this case raising the wine from the category of very good to that of excellent.

This is the kind of pretended objectivity that I object to. So there's a statistical difference between 89 and 90, because there is supposed to be some mystical (yet objective) difference between a "very good" wine and an "excellent" wine? In that case, is the difference between 90 and 94 less than 89 and 90, since I assume the former pair are both "excellent"?

Dale, Hi...

I see no "make-believe" or "pretend" here. What we are talking about are estimates in the eyes of a particular critic. Why is that any worse than: "An enjoyable wine; a great wine; a phenomenal wine"? Perhaps what some would like is nothing more than a "pass" or "fail" system? Even there though one might be curious to know if a wine barely passed or passed with flying colors. Remember the Blake Edwards film "10"? Bo Derek may have been a "10" for some. Not for me.

Methinks the anti-score people protesteth too much. Remember the Blake Edwards film "10"? Bo Derek may have been a "10" for some. Not for me. So should it be with both tasting notes/crits and scores.

Best
Rogov

What we are talking about are estimates in the eyes of a particular critic. Why is that any worse than: "An enjoyable wine; a great wine; a phenomenal wine"? Perhaps what some would like is nothing more than a "pass" or "fail" system? Even there though one might be curious to know if a wine barely passed or passed with flying colors.

As I said earlier, I have no problem with scores, grades, buy/don't buy, stars, or whatever (or whatnotever). But I do have problems with statements that imply there is an actual objective standard. "No statistical significance between 91 and 92 points. On the other hand, there is a major difference between 89 and 90 points" clearly implies there is a statistical significance between the latter. Which is bullshit.

With re professionals (and I agree that not all who write criticism and get paid for it are true professionals), I believe you will find major correlation between the scores of many of the better critics. That is not collusion on their part. It is that they have individually and collectively built up a repertoire of standards and those can be reflected in both tasting notes and scores. For sure there is sometimes wide variation but that is more the exception than the rule and in those cases one follows that critic that most often speaks to his/her personal taste.

I agree there is quite a bit of correlation of the scores of the major critics, and herein lies the issue I have. Many of the "Parkers" of the world - although they may appreciate and score old world wines appropriately - lean heavily in their preference to new world styled wines. Enormous, over-oaked, jammy reds, etc. That is all well and good if that is your preference, but it seems they overlook many old world gems because they don't fit into their ideal.

I have long trusted favorite importers over critics when choosing my wines, because I feel that their tastes also shine through in the wines they offer, and I can likely find some (Kermit Lynch is a great example) that fit my palate preference, and therefore I won't be disappointed.

As I said earlier, I have no problem with scores, grades, buy/don't buy, stars, or whatever (or whatnotever). But I do have problems with statements that imply there is an actual objective standard. "No statistical significance between 91 and 92 points. On the other hand, there is a major difference between 89 and 90 points" clearly implies there is a statistical significance between the latter. Which is bullshit.

No, Dale, it is not bullshit. Every grading or scoring system has watershed or breaking points that differentiate between levels of quality. In the case of 89 versus 90, for example, we are talking about the breaking point in level of quality as judged by the critic in question. With specific regard to wine, for example, that is why barrel tastings are always scored within a range and indeed that range often says, albeit in shorthand form, that a wine in the future may vary in qualityfrom what that critic deems to be excellent or extraoridnary.

I disagree with your point of view as you do with mine. I will not, however, say that you are spouting nonsense or bullshit. For heaven's sake, we are discussing a scoring system, not whether there is or is not an afterlife.

Best
Rogov

Many of the "Parkers" of the world - although they may appreciate and score old world wines appropriately - lean heavily in their preference to new world styled wines. Enormous, over-oaked, jammy reds, etc. That is all well and good if that is your preference, but it seems they overlook many old world gems because they don't fit into their ideal.

Joy, Hi Again....

Perhaps worth keeping in mind that a great many of the most respected wine critics in the world are not based in the United States and most surely do not appreciate the "enormous, over-oaked, jammy reds" and wines of that ilk. Indeed, not all critics, even in the United States have the power of Robert Parker. Regardless of their power, however, certainly not all appreciate or are biased toward such wines.

Best
Rogov

No invective here; just calm peer-to-peer discussion of concepts and ideas, Daniel.

But when I see this statement

In the case of 89 versus 90, for example, we are talking about the breaking point in level of quality as judged by the critic in question.

I have some doubt.

If you are operating on a clear, "precise" ordinal system, you can't put unequal weight on the ordinations you use. The nature of numbers is that they are all equal---therefore the space between the integers is also precise, so there is no allowance for the significance of "breaking points". Not in the numbers themselves; that can exist only in the subjective mind of the observer.

I see no possible way to posit an actual---versus projected---"breaking point" between 89 and 90. Certainly no more than there is between 88 and 89, or between 90 and 91.

The breaking point exists only in your mind, not in the numbers you use. You create and impose it, by putting weight on specific numbers. And that takes away any sense of objectivity.

I fully agree with your original statements (as you know I do) that humans are subject to numbering and ordination and ranking. But the use of a 100 point scale implies a precision that is not possible when considering wine (and that was my point when I put forth the three wines in my earlier question---which question I noticed you side-stepped with immaculate grace and dexterity to avoid answering . How can you possibly apply the same numerical rating scale to three entirely different and contrasting wines? I know we have a desire to rank things, but how can you rank those three in any meaningful way?

My antipathy to the point system, in your case, is due to another reason: you put great care and thought into putting together the right words to describe and detail each and every wine you critique? Why in the world would you negate that care and concern and artistry by reducing it all down to a simple number score, merely for the convenience of a lazy reader to whom such a wine is meaningless anyway?

Whether that box would be measured as attractive, as "comfortable" and as "well equipped" to take us on our voyage to wherever will be left for some pundit qua critic to decide.

95 (unoaked)



89 (pined)

resting note will follow.