Wine Talk

Snooth User: Philip James

Wine Scoring Methodologies

Posted by Philip James, May 6, 2008.

I've been fascinated with wine scoring methodologies ever since I stumbled acorss this fantastic post and accompanying pdf summary:
http://www.delongwine.com/news/2006...

In the two years that have passed since I found that I'm proud to say that I've actually met the author, Steve DeLong - founder of the Wine Century Club (http://www.delongwine.com/century.html)

The (ultra) basic history is this: over time the number of degrees of freedom of a wine score has increased from 3, to 5, to 20 (common in the UK still) to a whopping 100 (made popular by Parker) to several hundred (made possible by 89+ type scores).

People are beginning to ask if 200+ degrees of freedom is truly scientific. Its like me dividing a $100 dinner bill between 3 people and declaring its $33.33 recurring per person. Just because I can do it, it doesnt mean I should.

There's a backlash against this, driven in part by wine bloggers and consumer sites who use 5 star (glasses) type ratings, and by the very proponents of the 100 point systems who either say they only use the system because their competitors do (Tanzer) or that there's no difference in a 2-3 point band (Parker). Its also a slightly broken scale if the bottom 50% of it doesnt exist, and the bottom 70% is virtually never used...

Some useful links on the old guard scores:
http://www.erobertparker.com/info/l...
http://www.wine-searcher.com/wine-s...

Link to a meta rating (more like SnoothRank's aggregate rating):
http://www.winebluebook.com/lp/8/Wi...

And finally, thanks to a tip by RBoulanger, here's the most obtuse, yet densely packed with information rating I've seen:
http://sommelierjournal.com/about-u...

What about you guys? What wierd ratings have you seen, or which do you trust? How many 'degrees of freedom' for a scale is fair? 5? 10? 200?

Replies

0
0
Reply by Sung, May 6, 2008.

I don't think 200 really makes sense because are our tastes so refined that we even have the ability to score them to such a scale? Can I tell the difference between 180 and 182?
I like that Snooth uses the 1-5 point scale with the half glass ratings because as a novice, I can surely tell the difference between a 2.5 wine and a 3 point wine.

1
2
Reply by sdelong, May 6, 2008.

Hi Philip,

Thank you for mentioning this article - it's 2 years old but still seems mostly relevant. The big exception is that it was written pre-Snooth and at the time no wine site (I knew of) was using a five point Amazon.com type of rating. There were plenty of 5 star wine rating scales (notably Michael Broadbent and Decanter magazine) but in those ones 1-star is acceptable while at Amazon and Snooth, 1-star means bad.

It’s a great idea to actually use a system on the internet that web-users understand intuitively. I can see that you don’t have to go to any effort to explain it. Like it or not, your 5 star system is the way people know how to rate things online.

It will be interesting to see how things play out over the next couple of years. Will the web user 5 star scale of Snooth overtake the predominate 100 point wine expert scale?

1013
1382
Reply by Mark Angelillo, May 7, 2008.

I don't think I can effectively rate wines on a 100 point scale. I wouldn't sit down and compare two wines and give one an 82 and one an 83. Maybe for the super critics it makes sense because they have so much experience and can get that kind of nuance in there, but for me 5 glasses with half points is just fine.

2031
3202
Reply by Philip James, May 19, 2008.

I just came across this method, which doesnt actually score wines, so much as provide a graphical wave function of the taste:
http://www.spittoon.biz/tasting_not...

This out geeks even the sommelier journal system i mentioned initially.

2031
3202
Reply by Philip James, May 19, 2008.

And although I never understood it myself, there's always the "upload a photo that can act as a review of this wine": http://chateaupetrogasm.com/

Its a cool concept and works when you're using a gooseberry with a crown of thorns to describe a super high acid abrasive sauvignon blanc, but when the image is Harrison Ford as Indiana Jones I'm lost as how to interpret that.

0
0
Reply by Sung, May 19, 2008.

OK, Indiana Jones for the Lytton Springs, maybe that means like adventurous or something fun like that, but that weird picture of the dude who's face looks like a wood panel for the Titus Chardonnay is really freaky. I definitely don't get that one. Nor does that endorse the wine for me. It looks like a billboard for a horror movie.

800
288
Reply by Chris Carpita, May 19, 2008.

@Philip: The tasting note "innovation" is a pretty cool chart, but the idea of patenting something like that is silly. I can invent my own goofy looking box-with-dots system, thank you very much.

439
455
Reply by oceank8, May 19, 2008.


This weekend I checked out a new store:
http://www.vino100.com/
For someone who doesn't like or agree with the normal point system, this store is great. They rate wines on two scales: dry to fruity and light to full bodied. It is a great way to go in and find something new that you might enjoy.

2031
3202
Reply by Philip James, May 20, 2008.

Ocean, we have these guys on the East Coast that do it super simply: http://www.bestcellars.com/

They have 8 categories, and thats it: fizzy, soft, fruity, luscious etc

244
772
Reply by Rodolphe Boulanger, May 20, 2008.

I think the magic degrees of freedom number lies somewhere between 5 and 10. When Snooth was on a 5 full star scale (like Amazon or Netflix) I felt very constrained. It was hard to differentiate that you thought one 4-star wines was better than another, yet still not worthy of a 5. The current system works out pretty well. To be honest, I wish Netflix would allow half stars.

Although, for work I use a 200 point scale, it really works out to far fewer degrees of freedom that that - say 15 tops. Similarly, have you ever seen a wine rated 45 or even 55 by Parker or a magazine?

The more important question (and vino100 has caught some of it) is what are we grading the wine on? Performance? Value? Fruit expression? Fizz? Aging potential? Body? Intensity? All this leads to the ultimate question - why was this wine rated in the first place?

Too bad the answer doesn't lie between 41 and 43...

0
2
Reply by Phil V, Jun 6, 2008.

A little late to the party, I guess, but your description of our "rating" system really made me smile, as it is intentionally designed to be obtuse. Since you seem interested in these sorts of decisions, we decided that since our audience generally isn't fond of ratings, and being professionals who taste a lot, has less use for them, we wouldn't have them. But we also thought it was useful for our Tasting Panels to see how the wines stacked up against each other without being able to really pull a specific number out. If you want to see one of these Snapshots in action, here is our Tasting Panel from the (free) Preview Issue: http://www.sommelierjournal.com/art... (you need to click on the graph in the Gallery section, then click on it again to make it big). This is always the only place in the magazine where wines are "scored" in any way.

2031
3202
Reply by Philip James, Jun 8, 2008.

@Phil V - thanks for stopping by. You make a good point - you're catering to a certain audience with different needs and expectations than the average wine consumer magazine.

The first issue was December last year?

0
2
Reply by Phil V, Jun 8, 2008.

Well, technically it was a late October/early November issue, it works better technically on the website to have it as the "December" issue. Our first regular issue was the Premiere Issue published in April, our June issue is up on the site now (and in subscriber hands), and our July issue is about to go to the press.

244
772
Reply by Rodolphe Boulanger, Jun 10, 2008.

...and having read the first 3 issues of Sommelier Journal, I can heartily recommend it to anyone who wants a more serious approach to wine (no travelogues, no interviews with Madonna about her wine, etc.) and the wine business. Although it is targeted at the on-premise wine trade, it is interesting for the (advanced) consumer as well.

2031
3202
Reply by Philip James, Jun 10, 2008.

RBoulanger - i figured you'd have read it after you tipped me off about their wine scoring graphs


Back to Categories

Top Contributors This Month

324443 Snooth User: outthere
324443outthere
110 posts
680446 Snooth User: JonDerry
680446JonDerry
86 posts
847804 Snooth User: EMark
847804EMark
68 posts

Categories

View All





Snooth Media Network