Coffee Review is an online publication that reviews coffees and comments on them in formally written tasting reports and informal blogs. Kenneth Davids and Ron Walters founded Coffee Review in 1997. Today it hosts over one million unique visitors per year. All of Coffee Review’s past seventeen years of reviews, tasting reports and blogs are archived and available on the site.
Coffee Review originated the use of 100-point scale ratings for evaluation of coffee. The Specialty Coffee Association of America and the Cup of Excellence both began using 100-point systems about a year or two after Coffee Review began the practice. More on our use of the 100-point scale appears below.
Who Rates the Coffees and Writes the Reviews for Coffee Review?
Coffee Review editor Kenneth Davids writes most reviews, and all tasting reports. Ken has been in the coffee business for more than 40 years, and since co-founding Coffee Review has mindfully tasted and systematically evaluated well over 30,000 coffees. He has published three books on coffee that collectively have sold more than 350,000 copies.
Currently two additional trained tasters test all coffees with Ken and contribute evaluations and language to reviews. Associate Editor Jason Sarley’s sensory and descriptive skills, originally honed through tea studies and tasting with renowned San Francisco Chinese tea authority Roy Fong, were confirmed when he passed the Coffee Quality Institute’s Q-grading exam. Associate Editor Kim Westerman is a widely published food writer (New York Times, Forbes, many others) and certified wine sommelier. Guest tasters from the coffee industry also often participate in Coffee Review’s cuppings and tastings.
How are Coffees Evaluated?
All reviewing is done blind, which means coffees are identified only by number at the point they are cupped or tasted.
- Tasting protocols are standardized. Specialty Coffee Association of America (SCAA) cupping protocol is used for coffees intended for brewed application, for example.
- Although Coffee Review articles discuss and often foreground socio-economic and environmental issues around coffee production, and although articles may mention lifestyle issues, we feel that our unique contribution is honest and mindful sensory evaluation of coffee. We do not evaluate coffee on the basis of what is cool or fashionable, and, although we might enjoy seeing sustainably certified coffees score high, or underappreciated origins score unexpectedly well, or a small start-up roaster shine, we do our best not to let such personal preferences or whims influence us.
The Relativity of Taste (or: How Can You Tell Me that Coffee is a 94?)
I discuss the larger problematics of evaluating coffee using a 100-point scale in “The 100-Point Rating Paradox,” an article first published in Roast Magazine in 2010 and reproduced here. Below is a very much condensed, thumbnail version of the argument made in that article.
- Taste, or sensory preference, is fundamentally relative, and there can be no claim to universal validity attached to an evaluation of coffee or any other beverage.
- But we also can assume that there are communities of taste or sensory preference that reflect widespread consensus within a given culture or cultures.
- One such community of taste/sensory interpretation exists in the global community of fine coffee, encompassing coffee professionals and aficionados in both consuming and producing countries.
Here is Coffee Review’s understanding of those global coffee community assumptions:
- The acidy sensation in coffee is fundamentally good, so long as it is not harsh, overbearing, or astringent.
- Smoothly viscous or lightly syrupy/silky mouthfeel is better than thin, watery, or silty mouthfeel.
- Aromatic and flavor notes that are complex and relatively intense are better than those that are simple or faded.
- Given that coffee is an inherently bitter beverage, natural sweetness is good, whereas too much bitterness is bad.
- Aromas and flavors that develop naturally from the coffee bean itself when skillfully processed and roasted, like floral, fruit, citrus, honey, molasses, and chocolate, are better than flavors that come from mistakes made during fruit removal and drying, such as the taste of rotten fermented fruit, mustiness or moldiness, or medicinal flavors.
- A long, sweet, flavor-saturated aftertaste is better than a short, fast-fading, astringent or aromatically empty aftertaste
Both implicitly and explicitly, Coffee Review follows these assumptions in generating its evaluations and reviews.
Why Don’t I See More Dark-Roasted Coffees with High Ratings on Coffee Review?
We do our best to be positive about dark-roasted coffees, and there is usually a celebration around the cupping table when a darker roasted sample emerges with a relatively high rating. However, over years of mindfully tasting thousands of coffee samples we basically have had to accept that dark-roasting, or roasting much past what roasters call the “second crack,” does not produce a superior coffee when that coffee is evaluated using the “global fine coffee assumptions” cited above (some espressos may be an exception).
Many consumers enjoy and will continue to enjoy dark- to extreme-dark-roasted coffees, because, as I indicated earlier, taste is relative.
But, again, given the assumptions cited earlier and shared by most in the global fine coffee community, post-second-crack dark-roasted coffees do not appear to deliver the optimum coffee sensory experience obtainable from a fine green coffee. As the roast grows darker it mutes complexity and the sweetly bright sensation we call acidity, while gradually intensifying bitterness. Above all, darker roasting dulls the differences that distinguish various types and origins of fine coffee from one another, interfering with our enjoyment of those differences.
What about Extreme Light Roasting?
Although we sometimes cup a coffee that is simply roasted too light (tastes woody or starchy, with undeveloped aroma and flavor), in general:
- We have found that coffees can be fully developed at a quite light roast
- The key is the phrase “fully developed;” we occasionally taste coffees brought to a light but still reasonable roast color, yet these coffees are still, in our view, not fully developed – the cup remains little starchy or woody, or thin in mouthfeel, or grassy, or tea-like in a negative sense, etc.
- We assume that these coffees have been poorly profiled during the roast; above all, that the center portion of the bean has not been sufficiently developed
- It is true that a really fine green coffee will still, given our taste and calibration, score rather well at a slightly underdeveloped light roast. But we are often haunted by the feeling that the fine coffee we are cupping could have shown much better.
So that’s the Roasting Part. But Why Do Coffees that Are Roasted More or Less the Same Attract Such Different Ratings?
First, because some green, unroasted coffees are of a higher quality than others, and second, because among those higher-quality coffees some express their quality in more complex and interesting (“distinctive”) ways.
Some Green Coffees Are Better Quality than Others
Quality in green coffee is generally defined by how carefully the coffee has been handled in its long, complex journey from moist seed of a small, tartly sweet fruit to stable, storable “green bean” ready to be roasted. This complex story should be familiar to most readers of Coffee Review. Quality depends, for example, on how carefully the fruit has been harvested and sorted so that only beans from perfectly ripe fruit make it to the cup; on how carefully the fruit has been removed from the seeds or beans and by what method; and on how meticulously the fresh seeds or beans have been dried. If all of those demanding acts have been performed carefully, perfectly really, the coffee will be completely free of off flavors. Any distortions to its innate character will be subtle, pleasurable distortions that have been deliberately created by the miller (or selected by the exporter) to enhance the innate character of the beans.
Coffee that has not been treated well in all aspects of its preparation/journey typically betrays slight to dramatic off-tastes: a suggestion of rot or compost, for example, or mildew or mustiness, or sourness, or excessive bitterness. Although we frequently taste mildly tainted coffees at Coffee Review, we seldom review them because we feel that our main role is to identify coffees that consumers will enjoy and describe them in ways that convey that pleasure to our readers. We only concern ourselves with reviewing poor-quality coffees when we carry out surveys of broad market categories that do not allow consumers much real choice: branded supermarket coffees for example, or single-serve capsule coffees, or big-box store coffees. With high-end whole-bean coffees that offer consumers a broad array of choices we assume that our role is to identify and celebrate the winners, not castigate the losers.
Some High-Quality Coffees Are also More Distinctive, More Unusual, or More Complex than Others
Distinctiveness in green coffees is generated in many subtle ways: above all by the kind of tree that produced the coffee fruit, but also by nuances of fruit removal and drying and by the often mysterious impact of soil and climate.
Tree variety. This is not the place to testify once again that coffees from trees of the robusta species, however useful in certain limited roles, are not as interesting or distinctive as coffees from trees of the Arabica species, and that some cultivars or varieties of Arabica are more interesting or distinctive than others. There are stars and super-stars among the hundreds of varieties of Arabica grown in the world today, and, whether we want them to or not, coffees from these privileged varieties often dominate the very highest ratings at Coffee Review. They include the still rare and celebrated Gesha/Geisha variety, the various indigenous varieties grown in Ethiopia, the SL-28 of Kenya, and (slightly less distinctive but still likely to impress) the bean-beaned varieties Pacamara and Maracaturra and the ancient heirloom Bourbon and its derivatives. We are often embarrassed by how consistently coffees from these varieties dominate the very highest ratings at Coffee Review, but there is no fair way we can tilt the playing field against them.
Processing Method. Some coffees are more distinctive owing to the sensory impact of how the complex, subtle acts of removing the fruit from the beans and drying them (collectively called “processing”) are conducted. No one method of carrying out these acts (whether wet/washed, dry/natural, or honey/pulped natural) necessarily produces more complex, distinctive coffees than others. However, those producers who are meticulous as well as conscious about the impact of their processing choices typically deliver more distinctive, higher-rated coffees than those producers who are less caring and less mindful about them.
Finally, Back to Ratings and Numbers
The prevalence of high, 90-plus ratings on Coffee Review is the result of our policy of concentrating on coffee success rather than finding and punishing failure. We typically test forty to as many as 100 specialty whole bean coffees for a tasting report, for example, but only publish reviews of approximately ten, which almost always rate 90 or higher, sometimes much higher.
Also, we have found that the high end of the global coffee community has been relentlessly raising the bar for quality and distinction over the past ten years. We receive considerably more consistently fine samples now than we did even five years ago. So if the ratings at Coffee Review appear to be creeping higher, the reason may not be ratings inflation, but rather a reflection of intensified competition and achievement at the top end of the industry.
The higher end of our rating system currently calibrates roughly as follows:
97+ = Means: “We have not tasted a coffee of this style as splendid as this one for a long, long time”
95-96 = Perfect in structure, flawless, and shockingly distinctive and beautiful
93-94 = Exceptional originality, beauty, individuality and distinction, with no significant negative issues whatsoever
91-92 = An very good to outstanding coffee with excitement and distinction in aroma and flavor – or an exceptional coffee that still perhaps has some issue that some consumers may object to but others will love – a big, slightly imbalanced acidity, for example, or an overly lush fruit
89-90 = A very good coffee, drinkable, with considerable distinction and interest
87-88 = An interesting coffee but either 1) distinctive yet mildly flawed, or 2) solid but not exciting
85/86 = An acceptable, solid coffee, but nothing exceptional — the best high-end supermarket whole bean, for example
