The German Wine Tasting – Millman Objects


In a recent posting, I reported on the results of the German wine tasting of the Lenox Wine Club. The article was later picked up by Global Economic Intersection. Robert Millman who manages Executive Wine Seminars offered the following comment:

Professor Morss: I teach Philosophy and I run wine seminars. This particular tasting was, from both a methodological and sensory perspective utterly misguided. Why? Simple–you can never mix wines with different degrees of residual sugar in one tasting. The only meaningful comparison would have been between all dry Rieslings or all Kabinetts, or all Spatlesen, etc. Even the best and most objective tasters will tell you that once you have experienced a wine with noticeable RS, it is simply impossible to judge a dry wine even if it is made from the same grape, producer and vintage. So re-do the tasting with all dry Rieslings. Then you can see how the boxed wine really performs.

I was not particularly taken with the comment and decided to respond.


Professor Millman:

I am writing this in response to your comment on my German wine tasting results posted on Econ Intersect. I also looked at your wine seminar web page and was impressed.

Several points:

1. Philosophy – After despairing of comparative religion courses at Williams, I took a series of courses taught by Robert Gaudino. Gaudino was from the “Strauss School” at the University of Chicago; his Strauss reading assignments were tough. But that series of courses gave me meaning and structure for my life that I have followed ever since.

2. Regarding your comment at Econ Intersect, I found it a bit over the top and troubling. What made it troubling was your assertion – “This particular tasting was, from both a methodological and sensory perspective utterly misguided.”

You made this assertion without having any idea of our Club’s objectives. They are to:

  • Taste wines similar enough to make comparisons meaningful;
  • See if price matters, and
  • Have a good time.

Members of our wine club are experienced drinkers – all have been drinking wines 20-30 years. And by now they have at least some idea of what they like and dislike. At this tasting, our objective was to introduce the members to German Rieslings. And while our members are experienced drinkers, many have no real sense of good German wines. We wanted to find out whether they liked the fuller bodied sweeter wine or the less sweet ones. And the results came out in the tastings: the fuller bodied Spätlese actually won, but several preferred the lighter and less-sweet Bota Box.

You talk of the “best and most objective tasters”. I am wondering who these might be. My own sense is that wines are good enough today so individual preferences dominate, even among your “best/most objective tasters”. This comes out quantitatively in almost all major tastings when you look at the lack of correlation among scores as measured by, say, the Kendall Tau.

For example, consider the following summarized from an earlier piece:

1. Richard Quandt’s summary of the 2012 tasting at Princeton that was intended to be comparable to the Judgment of Paris:

“Four French wines were matched against six California reds in one tasting and four French Chardonnays were matched against six American ones in the second tasting. …on the whole the French reds beat the American wines, even though the single best wine was American.” There was no significant difference between the French and US Chardonnays because “while four of the five best wines were American, the two worst wines were also American, one of those by an overwhelming margin.”

2. George Taber drew a broader conclusion from the tasting:

“The Paris Tasting shattered two foundations of conventional wisdom in the world of wine. First, it demonstrated that outstanding wine can be made in many places beyond the hallowed terroir of France. Second, the Paris Tasting showed that winemakers did not need a long heritage of passing the wisdom of the ages down from one generation to the next to master the techniques for producing great wine.”

Several other widely publicized tastings comparing French and Californian wines were carried out over the next decade:

1978 (San Francisco), 1986 (French Culinary Wine Institute), and 1986 (Wine Spectator).

In sum and following from the Taber point, I believe all wines have become good enough so all judges are doing is registering their own preferences. And fortunately for the wine growers all over the world, these preferences differ.

Incidentally, I don’t know if you are familiar with how Robert Hodgson “rates” tasters. I quote from a piece that summarizes his approach:

“Robert Hodgson has his own winery and has been troubled by what appeared to be erratic ratings his wines were receiving from judges at tastings. As a consequence, he has been analyzing judge performance at the California State Fair for over a decade. The key result is that only about 10% of the judges are consistent in their ratings, and this 10% are not the same judges year to year. He concludes that competition awards have a major random component. To correct this problem, Hodgson has come up with a method to judge candidates. The key to his method? Have the candidates do blind tastings that include more than one glass of the same wine in each tasting. If the candidates do not score glasses of the same wine nearly the same, they are not competent to judge wines. Hodgson’s suggested overall scheme is quite rigorous: candidates must do four blind tastings of ten glasses each. At each tasting, there are three glasses from the same bottle. And for a candidate to qualify as a judge, the scores given on the glasses on the same wines must be ‘close’”.

In the Lenox Club wine tastings, we employ a scaled-down version of the Hodgson approach: we have two glasses of the same wine and see how far apart each taster scores them.

You also commented on how to obtain evidence on how box wines perform. We already have it. In the 7 tastings done before the German tasting, a box wine came in either first or second. And that was against wines tasting as much as $70. It was also true whether the wines were ranked or scored.

Another question: should wines be ranked or scored? At last summer’s meeting of the American Economic Association at Stellenbosch, Neal Hulkower argued for rankings while Dom Cicchetti made the case for scoring wines. In essence, Hulkower argues that scoring introduces too many arbitrary taster decisions into the judgments while Cicchetti argues that scoring allows tasters to register different intensities of likes and dislikes. We do both at our tastings and at least so far both “scoring” outcomes are the same.

3. I looked your tasting site. I found it very impressive except for one point: All the wines you taste are expensive. Maybe your clientele will not come unless you offer high-priced wines. However, economic analysis finds no link between price and taste. I quote from a piece I did on the subject:

Price – Taste Relationships

Goldstein et al analyzed data from 6,000 blind tastings – a lot of blind tastings[1]! I quote from their findings:

Individuals who are unaware of the price do not derive more enjoyment from more expensive wine. …we find that the correlation between price and overall rating is small and negative, suggesting that individuals on average enjoy more expensive wines slightly less….

Lecocq and Visser[2] analyzed data from three data sets totaling 1387 observations on French Bordeaux’s and Burgundies. They report similar findings:

When non-experts blind-taste cheap and expensive wines they typically tend to prefer the cheaper ones.

They cite the following anecdote:

Ernest Gallo, the patriarch of the family-owned E&J Gallo Winery in California (the largest winemaker in the world), recalls how, in the early stages of his career, he once sold wine in New York. He offered a buyer two glasses of the same red wine, the buyer drank the two glasses and asked for the prices of the “two” wines. Upon hearing that the first wine cost 5 cents per bottle, and the second 10 cents, the buyer declared he wanted the 10 cents bottle. The message behind this anecdote is confirmed by many wine auctioneers who have noticed that in the auction room higher wine prices act as a stimulant rather than as a deterrent, thereby reflecting that for bidders, part of the pleasure is apparently to know that a wine is famous and very expensive.

Goldstein et al report a similar finding from earlier research that both trained and untrained tasters favored wine they knew in advance was higher priced.

Overall, Lecocq and Visser conclude taste has very little to do with wine prices:

Our results indicate that characteristics that are directly revealed to the consumer upon inspection of the bottle and its label (ranking, vintage and appellation) explain the major part of price differences. Sensory variables do not appear to play an important role.

So I have a suggestion for your future tastings: pick up a copy of Robin Goldstein’s “The Wine Trials”, and include a few of his selections in future tastings. An added bonus of the book (I have the 2011 version) is that the first 70 pages include the best summary ever done on the relationship between taste and price, why people buy wine, etc. I hope you also consider including a “box”.

[1] Goldstein, Robin, Almenberg, Johan, Dreber, Anna, Emerson, John W., Herschkowitsch, Alexis, and Jacob Katz, “Do More Expensive Wines Taste Better? Evidence from a Large Sample of Blind Tastings”, Journal of Wine Economists, v. 3, no. 1.

[2] Lecocq, Sébastien and Michael Visser, “What Determines Wine Prices: Objective vs. Sensory Characteristics”, Journal of Wine Economics, vol. 1, no. 1.

The content above was saved on the old Morss Global Finance website, just in case anyone was looking for it (with the help of
This entry was posted in Drinking, Entertaiment, Featured Articles. Bookmark the permalink.