Snooth Blog

Snooth User: Philip James

Glasses Half Full

Posted by Philip James, Jun 28, 2007.

While in Napa we tasted a good deal of wine, of which there were some really excellent wines. It was at a Far Niente tasting that I realized that there's a big gap between rating a wine 4 glasses (like it) and 5 glasses (love it). Both Mark and I did find ourselves secretly wishing for a grade in between those (really like it?).

Don't get me wrong, we're big fans of the simplicity of 5 glasses/stars/whatever (and Pandora does a great job with just thumbs up/down), but the more degrees of freedom you get, the more you can express yourself. And I think ultimately thats the trade off - the rater gets the ability to rate with more nuance, but the reader gets proportionally more confused.

The jury's still out and I'm listening to users on the subject ( here , here , here and here are some bloggers reactions to the general idea of standardizing ratings), but half stars are an option, as is simply embracing the chaos and letting people rate in their native system - it'll be confusing to the reader, but they could adapt...

Thoughts?

Replies

0
0
Reply by Sung, Jun 28, 2007.

Would introducing a ".5" rating for users possibly introduce the same type of meaningless score, the way the 100 point scale does? I.e., what's really the difference between an 88 and an 89 point wine? What will be the difference between a 4.0 and 4.25 rated wine once the scores are calculated to create the SnoothRank?

Blog comment by Betsey, Jun 28, 2007.

My thoughts on this are along the same line as Sung's. If you are targeting the wine connoisseur, they would want and would be able to properly use a 10 point scale, but for the average/aspiring wine drinker a 5 point scale is more than sufficient, and anything more than that would accentuate the problem of each person interpreting the scale differently. You can outline for people that 4 means like it and 5 means love it, but can you really explain to raters what a 3.5 means and how it is different from a 3, in a manner that will ensure consistency across raters?

So I guess it really depends on who you are catering to, but I would say keep it simple and stick with a 5 point scale that forces people to rate within clearly defined buckets so that some sort of consistency is maintained as to what the ratings mean.

Blog comment by gr, Jun 28, 2007.

Especially since you display partial stars under SnoothRank, having the option to specify my rating in them would be nice. I think the glass icon is large enough that mousing over it in different locations can define the fullness of the glass, as long as you're only doing halves. (You'd want to do that from bottom to top, not from left to right, right?)

Also, I'd point out that your 5 point system's midpoint is "neutral", which leaves you only "like it" and "love it" on the right hand side. NetFlix's midpoint is "like it", which leaves them a slot available for "really liked it". I think their way is better. If I didn't like it, it doesn't really matter how much I did or didn't like it, unless it was truly horrible. I don't think I would ever be neutral. What's that even mean in this context? Either I will drink the wine again willingly, excitedly, or I'll go out and buy a case... or I will only drink it again grudgingly or I'll flat-out refuse it. What's neutral mean?

67
77
Reply by andrew, Jun 28, 2007.

I like the 5 star system for the reasons cited in comment #1, but continue showing the half stars in the display. I would shift the scale to the rating system proposed on Winecast since most wines I try are at least drinkable and usually just bland at worst.

* Not Recommended
** Average
*** Very good, Recommended
**** Delicious; A Wine of Distinction
***** Outstanding; A Classic Wine

So with the current system, I end up giving wines I'd never buy again a 3 star rating, because I consider them average and perhaps forgettable but I don't dislike them per se.

So here's my ideal rating shceme:
I'd prefer to give 2 stars to wines that are 'OK'. 3 stars is something I enjoyed and thought was a good value, something I'd consider buying again. Only a few wines make it to my list of wines I would go out of my way to buy again and I'd like to give these 4 stars. 5 stars would be reserved for the few wines that are really outstanding.
1 star would be the few wines that i did not like and would suggest avoiding.

1013
1382
Reply by Mark Angelillo, Jun 28, 2007.

I really like the idea of letting the user decide which rating system they want to use. It would be up to us to convert that to SnoothRank, which would still snap to the nearest half point of 5.

But half glasses would be a step in the right direction for me. I absolutely loved the Far Niente chards and their Dolce but the cabs wowed me slightly less. I really wanted those to be 4.5s, and it was really in the comparison that I noticed this.

67
77
Reply by andrew, Jun 28, 2007.

While I like the idea of a quality and value scale when reading about a wine I think average wine consumers would be a overwhelmed if given too many choices when they rate a wine.

Perhaps to recognize value or other characteristics in an easy-to-use way, there could be a few Digg style, 1 click 1 vote type buttons on the wine page, such as "excellent value" or seasonally appropriate buttons such as "Great Summer Wine".

Blog comment by Stephen, Jun 28, 2007.

I think the scale would really benefit from 1/2 points. When I taste wines, I usually use a 20 point scale, though I think 10 would probably suffice. For me the five glasses are
*never in a million years
**not for me (recognizing that it does have some features that others may appreciate)
***ok
**** Like it
***** Love it
In addition, in my experience using snooth's glasses, the biggest gaps are between 2 and 4. There are some that I feel are definitely better than ok, others that are probably less than ok, but above not for me. It is especially useful when you're at a tasting. If you have one wine with a meal and rate it then I think it's easier to give it 1-5 glasses, but when you taste dozens or scores of wines and you are comparing them all, then the limitations of a pure 5 point scale really become apparent. That is not to say more is always better, I think, practically speaking, that five glasses with half points give users enough flexibility without undermining the weight of the ratings.

Blog comment by gr, Jun 28, 2007.

I think Stephen and I might be saying the same thing (that I did in the second half, comparing to NetFlix's five stars), and I agree with his re-use of the scales.

I still say "neutral" is just a meaningless term and if you only want your scale to have one more point available for "very good but not excellent", drop neutral.

2031
3202
Reply by Philip James, Jun 28, 2007.

gr - thats good. its the easiest change and seems little disputed.

Blog comment by Stephen, Jun 28, 2007.

Actually I was advocating the use of 1/2 points, and I still think half points are useful. Someone offline pointed out that perhaps my distinctions are too fine for the average user, and I agree that a 20 point scale would be excessive, but I still feel that five points is insufficient, even on netflix. Repegging the scale to a different set of criteria still leaves holes. To use examples from things I have rated, I have two five glass wines, one would be a 4.5 the other a full 5. I also have a half-dozen three glass wines that range from 2.5 to 3.5. Changing what each of the glasses means does not address this number of variations, but 1/2 points obviously would.

Blog comment by el jefe, Jun 28, 2007.

Just make the scale logarithmic, that'll fix everything....

Or recognize that star ratings work great for peanut butter, but at some point (past your everyday plonk) they fall down for wine. Even for movies they end up meaningless: if "Die Hard" and "Citizen Kane" each get five stars, what does that mean?

It appears to be human nature to want to condense something to a single figure of merit, so it's probably hopeless. I'll be over here, by the windmill...

6
10
Reply by winehiker, Jun 28, 2007.

Philip, thank you for the "link love".

But Oh, the chaos! I agree that the 20-point scale is not for the fledgling wine drinker, and is useful mostly for those who would actively choose to expand their wine education. So if I would choose criteria for a 5-point scale, I feel that Andrew's and gr's concepts hit closest to the mark. After all, why waste points below a neutral "3" when wine buyers wouldn't particularly bother with less-than-average wines? Let "average" (i.e., every day drinking wines) reside at "2", and work the upper end of the scale from the above average throught the distinctive to the outstanding.

What concerns me is - since I intend to continue scoring wines on a 20pt. scale - can a wine rating on a scale of 20 be effectively aggregated and condensed via algorithm into a 5-point rating on Snooth? If so, this might be where the 1/2-point scenario shines best for us "Davis-scale" types who would broaden their audience to include the five-star crowd, i.e., an 18pt. wine counts as a 4.5, while an 18-1/2 pt. wine counts as a 5. (To my left brain, anyway.)

Thoughts?

2031
3202
Reply by Philip James, Jun 29, 2007.

12+ comments. i feel thats a record on this humble blog.

El Jefe (welcome) makes a good point, which is that the numbers are only a single dimension and if we trade 90 point parker wines for 4.5 point snoothrank wines we haven't achieved much. Now, Snooth will have a lot more information on each wine than the score - type, varietal, region data, user tasting notes, awards won, critics notes etc etc.

When we add in user voting (hey i found this review helpful) for the notes it'll become even more useful. but, not everyone reads the instructions, so the score is there for them just in case.

Winehiker (welcome too!) i think the changes we're leaning towards today are the moving of the neutral point to 2 (hate/dislike/like/really like/love) - this is the same as netflix, winecast and gr's request. And then adding half points. If people choose not to use them thats fine too, but in vertical or horizontal tastings its very hard to achieve enough degrees of separation with just 5 choices.

As for programatically converting scores, we'll do it, but its always a little icky - is decanter's 20 point scale the same as UC Davis's? Wine Enthusiast's 100 point scale is most certainly different from Robert Parker's...

Blog comment by el jefe, Jun 29, 2007.

"I found this review helpful" is a great thing to do - I think you'll find it gets used more than you might think...

Blog comment by Snooth Blog Rating changes, Jul 24, 2007.

[...] getting lots of feedback on the issue of ratings and half glasses, we have decided to follow WineCast’s system (with 2 [...]

5
1386
Reply by amour, Mar 28, 2010.

No one seems to be complaining.

Or ,  is it that no one thought about it recently?


Back to Categories

Top Contributors This Month

324443 Snooth User: outthere
324443outthere
132 posts
680446 Snooth User: JonDerry
680446JonDerry
93 posts
847804 Snooth User: EMark
847804EMark
68 posts

Categories

View All





Snooth Media Network