This is a revision of an article first published by Coffee Review editor Kenneth Davids in Roast Magazine in 2010. We offer it here as a considered overview from Ken on 100-point ratings systems for coffee and on the problematics and philosophy of coffee evaluation.
In 1997, Coffee Review started reviewing coffees for consumers and the trade using a 100-point scale. Such ratings were widely in use at the time for wines and cigars, but no one had used them before for coffee. At first we took some flak from coffee insiders, but not much. The industry seemed ready for the idea, even enthusiastic about it, and today 100-point ratings saturate specialty coffee communication, particularly within the trade, where they play a role everywhere, from green coffee competitions to green dealer reports.
Nevertheless, I harbored some misgivings about the 100-point system from the day I first used it, and I still harbor some. But ultimately I think 100-point-rating systems have exerted a tremendously positive influence on the coffee industry.
Numbers as Language
The drawback to 100-point systems, of course, is that they use a language we associate with objective measurement – numbers – to embody judgments that clearly are neither objective nor universal, but are deeply influenced by all the characteristics that make us human and hence both the same and different from one another – our culture, our personal histories, our sensory associations, our training, our own peculiar body chemistries, even the number of bumps on our tongues.
There is no such thing as an objective sensory reading, by the way. What the more scientific among us are looking for are reliable, repeatable sensory readings, readings that are reasonably the same time after time given the same set of sensory stimuli. But the associative structure that generates those readings can never be considered objective. I don’t want to go there, but even machines are not objective, since they only measure what humans want them to measure and the only meanings they generate are the meanings humans assign to their measurements.
Contrasted to the language of numbers, the language of the written word makes much less pretense to objectivity. When cuppers use text to elaborate and support ratings, I suspect readers understand that they are reading relative judgments that issue from actual human beings, rather than from machines. Text also allows judgments to be nuanced, giving at least a little culture, flesh and history back to the naked numbers of a rating.
Nevertheless, the number remains, towering over the fine print, tyrannical in its pretense to certainty, dominating first impressions. And in many other contexts – green coffee competitions, for example – there may be no fine print at all to nuance ratings: we get nothing but names and numbers, monumental and unassailable.
The Expedient Argument for 100-Point Ratings
How do we reconcile the pretense to universality and objectivity implied by a ratings number with its obvious origin in human relativity and subjectivity?
I think there is a sound philosophical basis on which to reconcile this apparent contradiction; more on that later. But first we might ask: Why bother with 100-point ratings for coffee in the first place? From an expedient perspective, what is their usefulness?
Very broadly, they are simpler and more dramatic than words.
In our culture, a ratings system dignifies fine coffee. A ratings system says to consumers that coffee is not just a matter of regular vs. decaf, but a thousand (OK, a hundred) degrees in between. Read the fine print for the details, perhaps, but the number makes the main point – that there are subtle but important differences, worthy of careful scrutiny, that separate one “regular” coffee from another.
Coffee growers in particular have benefited from the discriminations introduced by ratings systems. Certainly competitions, reviews and the Q-system have played a crucial role in calling attention to individual producers or producer groups who consistently produce outstanding coffees. And when coffees from heretofore obscure origins soar with consistently high ratings, this success focuses awareness on entire regions or countries that may have been overlooked by the traditional hierarchies and menus of specialty coffee.
In other words, ratings, and the blind tastings that generate them, are a means through which quality and distinction can be recognized on the basis of merit, rather than on the strength of tradition, public relations firms, or the sheer luck of attracting the attention of journalists more interested in the drama of a story than in the distinction of a beverage.
Ratings also introduce the tension and drama of competition into the coffee arena, and competition, like it or not, is a prime driver of engagement and respect in contemporary American culture – not to mention a valuable prod to excellence.
A ratings number is also a way to synthesize responses of a larger group of people – like a green coffee jury – while testing the consistency and reliability of those responses. At the same time, it is a way of provoking dialogue and exchange within the responding group. If it is just a question of your response versus mine, well, no problem, we nod and move on. But if we both need to settle on a collective judgment embodied as starkly as it is in a number, we may find ourselves usefully testing one another’s perceptions and conclusions.
Finally, and I need to tread gingerly with this one, I think ratings help consumers with a short attention span find their way through the rich but confusing maze of names and claims spawned by the specialty coffee industry. True, ratings are only a starting point on what is hopefully a journey of discovery on the part of consumers, but many consumers who care about our beverage find it a very valuable starting point.
A Way Out of the Subjectivity/Objectivity Bind
These are all justifications for 100-point ratings system based on expediency, you may object. They are not soundly based on “science.” They’re not even based on astral charts or I-Ching diagrams. Just a bunch of opinionated dudes and dudettes slurping, spitting and proclaiming.
As I suggested earlier, I think that there is a way out of the subjectivity/objectivity dilemma that is philosophically honorable, yet opens the way to legitimate evaluation of coffee. A couple of decades ago in the world of literary theory, a scholar named Stanley Fish popularized the following set of ideas, presented here translated into my own understanding. (Be patient with this; I believe it is an argument worth following.) Fish starts with the contention that everyone reads a piece of literature differently; in fact, there are as many different versions of Mark Twain’s famous novel Huckleberry Finn, for example, as there are people who have read the book. Not only that, but there is a new version of Huckleberry Finn created every instance the same individual reads the book again. Extending that point, there are as many different versions of the Beatles’ song “Let It Be” as there are people who have listened to it and moments when they listened to it, since different occasions always change our perception. The version we listened to half-drunk at a party will be different from the version we listen to when sober doing the dishes two weeks later, even if the artist and the recording are exactly the same. Our emotions, focus, body chemistry, and a lot of other things were different at the party than while doing the dishes.
Applying this idea to coffee, one could therefore argue that there are as many versions of a given coffee as there are people who have tasted it and moments when they have tasted it. All is relative, from person to person and even from moment to moment.
Nevertheless, Fish’s argument continues, there exist in societies “interpretive communities” that together use similar language and make similar assumptions about phenomena their communities define as related or the same. Literary critics, for example, are all members of a similar interpretative community, and although their specific interpretations of Huckleberry Finn may be different, sometimes quite different, nevertheless they all operate inside a broad set of basic assumptions that are accepted as “true” by all of them. Hence, when they read Huckleberry Finn, their experiences of the book overlap sufficiently so they can argue about it as though they really were all actually reading the same book at the same moment in the same state of mind.
A Global Interpretive Community
Similarly, a dominant global community of interpretation exists around coffee, and it is this dominant global community of interpretation and its shared assumptions that could lead us to accept the validity of a 100-point rating of a given batch of coffee as a legitimate act of evaluation and communication. Other communities of interpretation for coffee exist that make somewhat different assumptions about coffee using somewhat different languages (those who buy rio-y, medicinal-tasting coffees for blends in the Middle East and Central Europe, for example, or Europeans who enjoy espresso blends based on fermented natural Robustas). However, these communities don’t make much fuss about their assumptions and tend to keep their heads down, busy producing coffees the consumer members of their communities enjoy while not rocking the boat of the dominant group of coffee experts. (The dominant community of coffee experts I am describing is of course a global community, including coffee professionals from all over the world, with a particularly rich representation from Latin American coffee-producing countries.)
What are the criteria for excellence applied by this community of expert tasters?
- Acidity is fundamentally good, so long as it is not harsh, overbearing or excessively astringent.
- Smoothly viscous or lightly syrupy/silky mouthfeel is better than thin, watery, or silty mouthfeel.
- Aromatic and flavor notes that are complex and intense are better than those that are simple or faded.
- Given that coffee is an inherently bitter beverage, natural sweetness is good, whereas too much bitterness is bad.
- Aromas and flavors that develop naturally from the coffee bean itself, like flowers, fruit, citrus, honey, molasses and chocolate are better than flavors that come from mistakes made during fruit removal and drying, like fermented fruit, mustiness or moldiness, or rotten or medicinal flavors.
- A long, sweet, flavor-saturated aftertaste is better than a short, fast-fading, astringent or aromatically empty aftertaste.
This set of assumptions and the interpretive community around them existed long before anyone proposed 100-point ratings systems. The recent development of training/credentialing programs referencing the 100-point system, like the Coffee Quality Institute’s Q-cupper program, simply institutionalize and promulgate broad assumptions long shared by the dominant interpretive community for coffee.
The Interpretive Community and Consumers
A crucial question arises next: To what degree do consumers share the assumptions of this interpretive community?
In large part, Coffee Review’s mission is educating consumers about fine coffee. And a major component of that education, to put it bluntly, is getting consumers to taste coffees in the same way the expert members of the coffee interpretive community do; in other words, encouraging them to appreciate and apply the criteria for coffee excellence listed earlier. I would argue, based on feedback at consumer presentations, formal consumer studies, and conversations with coffee enthusiasts, that consumers mainly share most of these “expert” criteria, though with a couple of significant divergences.
The first and often most obvious divergence of preference between the expert community and many consumers is the issue of acidity. Some consumers not only do not like highly acidy coffees, but their bodies don’t like them either. The second area of divergence has to do with the greater degree of tolerance among some consumers for the flavor impact of certain taints. The current fad for big-fruit, sweetly fermented dried-in-the-fruit or “natural” coffees is not the result of a plot by wild-eyed millennial third-wavers to undermine the integrity of the international expert community. In fact, a lot of consumers like big-fruit naturals; they like them very much.
At Coffee Review, we try to honor both of these mild exceptions to the global coffee expert consensus. We are very aware that the specialty coffee industry was built on two pillars: quality and differentiation. From the very beginning, specialty offered the consumer not only better quality coffee, but more kinds of better quality coffee: a variety of sensory styles, types and profiles that engage the consumer as connoisseur as well as straightforward coffee drinker.
Disciplined but Open
How did I get here from my initial defense of 100-point systems? Simply as a way of contending that blind cupping and the reviews and ratings it generates should be seen as a way of testing and exploring the consensus of the expert community in a disciplined, ongoing process of communal definition and discovery, rather than as the detached application of an eternally fixed set of objective judgments. For me, blind cuppings and 100-point ratings systems should not only promote a disciplined, world-wide system of evaluation, but also and, perhaps, more importantly, provide an orderly and deliberate context for ongoing criticism and refinement of that system.
About this Article
This article first appeared in Roast Magazine in 2010 and is reproduced here with the kind permission of Roast. A section of this article also incorporates material appearing in Kenneth Davids’ contribution to the anthology Coffee, Philosophy for Everyone: Grounds for Debate edited by Scott F. Parker and Michael W. Austin, 2011, Wiley-Blackwell, Malden, Massachusetts.
