2. Do you have a standard review process that you follow and can you share it here? What makes your reviews different compared to other reviews you have seen?
Candice Alstrom I take a while to do a review. I try to capture every nuance I can. I am still an amateur and there is still a lot I do not know. But I try to give each bar as much time and respect as I can. I created a review system as basic as I could so that anyone could follow what I was doing. I based my grading scale on 5 areas for review and gave it a number system based on simple high school grading. It's all subjective grading which is another point I try to drive home. My taste is different from your taste and the reviews are strictly from my palate and perspective. I also try to be as honest about the conditions in which I am tasting. If I sat on a bar of chocolate longer than 3 months I indicate that in my reviews. A lot of times, I take detailed notes on bars and don't blog it right away. I will indicate that as an archived review. Transparency is key for me and helps the chocolate makers understand exactly what the circumstances are for my reviews. But generally, I open the bar of chocolate and begin to review it right then and there.
David Arnold My reviews were usually personal ones and not generally seen by others. Daily morning tastings with detailed notes were a way to consider regions, makers, styles, ingredients and all that went into a fine chocolate bar. It’s a great exercise to master. Currently there is less note taking but I spend more time with the chocolate. Some of my thoughts on taste have shifted. A few years ago I would have discounted emotions influencing taste and now I see nearly everything affecting it. My favorite way to get to know a bar is to taste it throughout a day and note how different it can be from one hour to the next. The chocolate isn’t changing but one’s enjoyment of a bar can be greatly enhanced with the right situation.
Blind tastings in a quiet room with a clean pallet will always have their place. Finding chocolates that hold up and wow you at a particular moment is what I’m looking for.
Should we avoid all bias? Shouldn’t we listen when a maker tells his or her story, philosophy, thinking behind the bar? Should a chocolate review not consider the package? I never want a bar I can’t easily open or one that isn’t protected properly. More reviewers should also be critical of claims that are made. I want to take everything into account but narrow my focus to what’s going on with the bar.
Debbie Ceder Reviews take a lot of time and consideration – far more time than we have to spend, I’m afraid. In general, our process is the following: Vic takes 4-5 bars of similar composition (origin, percent cacao, region of production, flavored or plain, etc.) and rates them consecutively before giving them to me for my rating. Each bar is tasted and rated based on a 1-10 scale (10 being perfect). We don’t have the time to spend writing up the tasting experience and the flavors encountered. Most times, I am rating the bars at work and it takes me a bit of concentration just to experience the bar and make a 1-10 rating alone. If the finish is long, it takes quite a while before I’m ready to taste a second bar, and I may not even get 4 bars tasted and rated in a day. I envy people that have the time and talent to write up detailed reviews of chocolate.
Vic Ceder We do not follow a strict process when reviewing chocolate, nor do we have blind tastings. Our process is simple: Each week, I select 4 bars with similar characteristics, selected primarily by origin, then by percent cacao. When tasting an adulterated bar (i.e., with milk, mint, or other additive), then I try to make sure that all four bars for the week's tasting are adulterated bars. In general, however, we avoid such bars. We have no desire, for example, to try a bar with bacon in it. Although I love eating mole at a Mexican restaurant, I loathe chile in my chocolate bars.
We prefer bars that are single source origin, in the 70 to 75 percent cacao range.
Bars are rated on a scale of 1 (intolerable) to 10 (stupendous), simply based on how well we like the bar. Occasionally there’s a bar that is so bad that I have to throw the rest away. Bars rated 1 to 2 are usually given away after the tasting; bars rated 3 to 4 are just barely edible; bars rated 5 to 6 are average to good; bars rated 7 to 8 are very good to excellent, and I don't hesitate to buy them for recreational snacking whenever I encounter them. To be rated a 9 or 10, the bar must be outstanding. A 10 is reserved only for those bars that are most exceptional, usually in terms of outstanding flavor. I'd go out of my way to obtain a bar rated 9 or 10.
Our database has a flag for 'Love at First Bite', which is reserved for bars that are unique in some manner (such as taste, texture, or additional ingredient) that we immediately recognize upon the initial tasting as being something special. For example, the Dolfin "Lavande Fine De Haute-Provence" bar falls into this category because I actually like lavender in chocolate and the bar contains bits and pieces of the herb. The Rogue "Jamaica" bar falls into the 'Love at First Bite' category for exceptional aroma and taste.
When tasting a bar, I proceed as follows: ... First I cleanse my palate by taking a few drinks of bottled water. I unwrap a bit of the wrapper, and bring the bar up to my nose for a close sniff. I break off a small piece, and examine it for bloom and texture. I then place the piece in my mouth and let it start to dissolve while discerning the taste. After about a minute, I usually chew up the remainder of the piece, for a big flavor burst, then let the chocolate completely dissolve. Then I wait for at least five minutes or more, to experience any aftertastes. Upon completion of this process, I gather my thoughts and make a rating determination. If there are more chocolates to be tasted, I'll cleanse my palate, wait a few minutes, and then proceed with tasting the next piece. If I didn't get a nice assessment of the chocolate, I’ll do a second tasting of the same bar.
As there's no blind tasting, I suppose there is a chance I'd be more biased when tasting, say an Amedei Chuao bar, over some other random bar.
ChocoFiles I was trained as an engineer, so I apply many of the techniques of science experiments to my reviews. There are many variables that in the review process, so I have sought to standardize as many of those variables as possible. That standardization allows for the most meaningful comparison of each characteristic and then of bars as a whole. Here are some of the elements in the process that I have sought to standardize:
* Time of day- I have found that my palate is the freshest in the morning, so I begin each day by rinsing my mouth with some room temperature water. (Room temperature works best to standardize the taste buds. If the water is too cold the taste buds don’t register flavors as well and if it’s too hot they might get burnt.) Before I eat anything else the first thing that I taste is chocolate.
Occasionally I will do another tasting in the middle of the day, but when I do I make sure that I haven’t eaten anything for several hours so that my palate is a clear as possible. I don’t review more than one chocolate in the same time interval. In my experience the second chocolate will never get a fair hearing because the palate has become at least slightly dulled.
* Environment- I sit in a quiet place where I can really focus with no other distractions. I often close my eyes to concentrate as much as possible on the flavors.
* Temperature- this may be the most underrated factor by most reviewers but the temperature of the chocolate has a huge effect on the experience. If the chocolate is too cold then the mouthfeel is too hard and the flavors release too slowly on the palate. If the chocolate is too warm then the flavors release too quickly and the mouthfeel is too soft, sometimes even cloying. So in warm weather I try to only review chocolate that has been kept in a cool dark place. In colder weather I first put the chocolate in my pocket to warm it up.
* Sample size- I try to eat 6-8 grams at a time because keeping a standard amount creates approximately the same amount of flavor release. It the piece is too small then there is not enough flavor.
* Aroma- I first rub the piece to release more aroma, then I usually hold my left hand over the top of my nose to block out other smells. I also take a few breaths with my mouth open to allow the aromas to
completely circulate through the mouth, nasal cavity, and nose. This small act greatly intensifies the aromas experienced.
* Flavor and aroma descriptors- I combined a number of different tasting wheels and lists to make a list of descriptors that has broad categories and specific examples. It is quite tricky to translate aromas and tastes into other more familiar items and then to hope that other people understand them in the same way that you do.
* Notes- in order to remember as much as possible I created a database, and I take notes on 10 factors (that are described in the next question).
* Ratings: for the final step I give ratings. I do not attempt to avoid bias; instead I acknowledge that my ratings are subjective. My ratings are simply a measure of my personal enjoyment of a chocolate bar, so they based solely on what I enjoy, according to my experience and my personal preferences. Tasting chocolate is an aesthetic experience, so a great deal of subjective preferences and experiences will always be involved with it. My enjoyment Ratings are NOT intended to be an objective rating of the quality of a given chocolate. There will probably be a general correlation between quality and my preferences, but I believe that it is an impossible quest to look for a chocolate that can be objectively quantified as “The Best”. Instead, my much more achievable aim is to find “My Favorites”.
I rate on the curve by comparing a bar to every other bar that I have reviewed. I use a scale from 0 to 10 where 5 is average. When assessing a chocolate I start at the baseline of 5, and then I determine how high or how low the rating should be from there. I utilize the entire range of values, so anything above 9 is among my Top Favorites, 5 is average, and anything below 1 is among my most disliked. Keep in mind, too, that since chocolate is an agricultural product bars may change, so these ratings are only for a specific vintage of bar at a specific point in time.
Class Rating: I give each bar 2 ratings-- a Class rating and an Overall rating. My Class rating is one difference in my ratings from those of other reviewers. I’ve separated chocolates by “Class” because there are so many different kinds of flavors that are very different from each other. For example, White chocolate and Spicy dark chocolate are so different from dark 70% chocolate that I don’t think that they can be meaningfully compared. Even 55% chocolate is so different from 99% that they are very hard to compare. The purpose of the Class Rating is to be able to compare chocolate in a similar category, so that when I’m in a mood for a certain flavor I’ll know what I like most in that class. For each Class the highest rating is 10, even if that particular class is not one of my favorites. As an example, I don’t really care for nibs, but within the Nibs Class there is a bar with a Class rating of 10, even though the
Overall Rating is only 4. I interpret this to mean, “If I wanted a nibs bar I’d choose the 10, but overall I enjoy nibs below average, so I usually won’t choose a nibs bar anyway.” The Classes include Dark in 50s, 60s, 70s, 80s, 90s, 100%, Cherry, Coffee, Ginger, Mint, Lemon, Milk chocolate, Nibs, Nuts, Orange, Other, Pepper & Salt, Spices, Spicy.
Overall Rating: Then I also give an Overall rating that is based on my preferences, rated on the curve, and compared to ALL chocolate that I’ve tasted. This rating generally follows my hierarchy of preferences from the various “Classes”. For example, I prefer bars in the range of 70-78%, so the best of those bars have my highest Overall ratings.
Quality Grade: The quality of a chocolate bar is very difficult to quantify, and there will always be an inescapable element of subjective bias in doing so. However, after reviewing over 650 chocolate bars I think that I can generally determine the quality of a chocolate bar. In the broadest terms there are 3 categories: Good, Average, Bad, although there is a gray area between each category. I have expanded on those categories a bit by using a school style grading system that uses five grades of A, B, C, D, F. The grading scale starts with C being Average and goes up or down from there. Because of subjectivity and the many challenges to rating quality I think that it is too difficult to be more precise than this.
Putting it all together, after I have done each step, and slowly pondered each element, it usually takes me about 4-7 days to review one bar. The final step is to post my review notes to the ChocoFiles website.
Mark Xian “Do you have a standard review process that you follow and can you share it here?”
the C-spot® process is very transparent in each of its reviews by their very layout & design, & how each aspect is weighted.
As per the timing, setting, logistics:
Storage -- 65ºF / 55-60%RH; chocolate aerated for approximately 2+ hours & brought to ambient room temperature of ~76ºF
Some sessions are conducted collegially at roundtables to compare notes, while others are done individually. Whenever the latter, the first encounter is tested blind, put out by a second person, without any wrapper in sight or markings on the mold.
Once the initial evaluation is recorded, I then give it a whole day, resampling intermittently to re-confirm, or alter as the case may be, those first impressions.
For special cases of truly monumental impact, & therefore import, it can extend to a whole week in order to really ‘get to know it’ & get a clear read on it. Unlike coffee cuppers, or wine sommeliers who shotgun thru cases of bottles if not barrels at a single sitting (Parker racks 10,000+ a year), chocolate delimits itself by the melt which requires more time, leisure & consideration for proper evaluation as opposed to a swig & spit & taste-notes of “big fruit; score it a 10”. Chocolate, after all, embodies a full sensory experience -- often an overload -- engendering intimacy. So we ‘chew’ & ‘melt’, between alternate tastings, because both have their strengths &, moreover, each reveals a different side of flavor.
Which leads to the question of ‘when’. Some “gurus” proclaim that certain times of the day are more optimal than others for critical tasting periods. They’re usually the same ones who brandish antiquated “taste maps” that divide the tongue into regions, each corresponding to specific sensations of sweet/sour/ bitter/salty & the newest member – umami. In reality, the tongue possesses manifold & indiscriminate taste receptors wired to the brain.
Our scent-limit for discerning a variety of aromas at any given single instance has been scientifically measured at around a half-dozen, except alpha-tasters can quadruple that number in a game called ‘Sniffy-Sniff’. Here’s how to play along at home with chocolate:
Always best, they say, to conduct ‘formalized tastings’ first thing in the morning… even if nature calls with runs to the bathroom. Start by picking a scent – could be any scent – let’s see what sticks to the palate walls… for instance ‘this bar bathes the taste buds as a bar of soap does the skin & smells like yummy hair spray atop an excretion blown out to a sea of mouthwash… so if you’re feeling kinda Minnesota, this will make you look Californicated’. Then pass out ratings based on the most severe grade-inflation (kissing ass helps procure free product from chocolate makers in the future): 9.9. Yeah, that’s a good score.
All just a minor aspect of the maker-critic incest. ‘Gee, one of the great things about this job is all the free chocolate.’
Now that you know how to play the game, let’s examine the fetish of ‘only-early-in-the-morning’ for a proper tasting session. People liken chocolate to sex since it contains traces amounts of aphrodisiacs (such as PEA – phenylethylamine, the “Love Chemical”, an adrenal-related compound that is also naturally created within the hypothalamus of the brain & is released to flood the brain during orgasm, producing that warm glow).
Some humans enjoy “quickie sex’ first thing in the morning, right before rushing out of bed, mainly on account that even rapid faster-than-a-blink sex is better than none at all seeing how many have to bolt for the door & get to their 9-5 cubicle.
But is it the best sex?
Maybe; occasionally; especially to those who prefer groggy, blurry sensations.
Then again, midnight sex can be pretty orgasmic too. Or the outdoor variety under the midday sun at noon when the senses are fully awakened. (By way of analogy, shoe salesmen recommend trying on a new pair in the afternoon when the foot is more likely to be fully swollen.)
Ditto spontaneity as opposed to some monastic ritual of keeping the hours with prayers.
Truth be told, whenever the moment seizes & moves, be it 3 in the morning or 4 in the afternoon, carpe diem in sync with Roberta Flack “that’s the time, I feel like makin’ love to…”
Besides, such an M.O. more faithfully enjoins the slow food movement, which many choc specialists advocate. Oh, which reminds me… an overall clean, simple diet is primary for a sharp palate.
“What makes your reviews different compared to other reviews you have seen?”
This, again, is self-evident.
the C-spot’s® reviews delve into the matter, conveying the most comprehensive in-depth information – both technical & descriptive -- bar none (bad pun intended).
It does so in such a way that if all you want is the quick gloss / overview with numeric ratings & hi-, medium, low metrics on a bar or boxed assortment’s essential qualities, those are easily indexed at the top of every review page. Plus the data points, of who, what, when, & from where, as well as the genotype (varietal, cultigen, etc.) are also easily indexed.
Alternatively, if readers desire to bite into deeper layers, the expository sections of the reviews deliver copious explanatory notes for them.
Above all, the C-spot’s® reviews lead with an Impact statement. Usually a free-style verse & often irreverent, the Impact Statement relates the personality of a chocolate. Every bar & box has one. A few comment that our “vivid metaphors” go too far. Well, people may want the clinical facts but we’re hard-wired for storytelling too.
For example, the Mayans, chocolate’s great primogenitors, placed cacáo at the very center of their ideology as a sacrament where the deified was reified & vice-versa. Their venerable history still holds considerable importance for them right up thru to the present.
A defining quality of the Mayan cosmo-vision is a cross-media, sensual dimension that animates objects, landscapes, myths, deities… in reality everyday life. For the Maya, the world was a transformational, multi-sensory place, governed by analogic reasoning where all senses – sight, scent, touch, hearing & taste – merged in what Mayan scholars Houston & Taube call ‘cultural synaesthesia’. Set within landscapes the Maya conceived as sacred, substances such as jade attracted moisture thru magnetic force, bestowing greenery & fertility to the fields surrounding it. Similarly, turquoise, the property of the gods, was believed to emit smoke, while quetzal feathers symbolically linked those gods to Mayan rulers. And cacáo with its association of heart ‘n blood would literally course thru the arteries of trade networks as a type of currency.
Less than a century ago, cacáo continued to circulate as a form of currency in Central America. Even today it solemnizes weddings, birth rites, Spring fertility festivals, healing ceremonies & generalized cofradías (brotherhood). Cacáo especially retains sacredness during Easter — its association with “re-born again” ancestors now transferred to the crucifixion on which it resuscitates the Christ for his resurrection.
These manifold symbolic & metaphoric associations of the Maya were picked up by Euros who confiscated their goods to hand them down thru the generations. Several continue to linger in the West to this day, such as the link between chocolate & romance.
In light of all this, the C-spot® Impact Statements, hyperbole & all, might be too tame by comparison.
Language represents another area that set our reviews apart. Up until recently, a vocabulary specific to premium chocolate did not appreciably exist other than copycats stealing terms from oenophiles or, worse, rote foodie talk which usually boils everything down to ‘tasting-by-number’ & meaningless spittle (hence the C-spot® List of Banned Words). There is still a long way to go on this but the C-spot® basically devises a nomenclature in the spirit of elevating the field.
We’ve coined several terms:
Barsmiths – refers to those who work from the raw material cocoa; replaces the clunky-sounding “bean-to-bar” chocolate makers (bean assigning a humble if not altogether cheap connotation right up there with ‘pinto’ & ‘fava’)
Bromans – we never address the people who cultivate cacáo as “farmers” because, again, farmers in the English-speaking world are considered dirt-poor rubes. Winos never call their farmhands that pick grapes “farmers”. Oh, no, they’re “vintners” & consumers pay them handsomely for it. the C-spot® prefers ‘growers’ or ‘Bromans’ in reference to those who husband Theobroma cacáo trees (the botanical name of the tree from which all chocolate derives)
Chocorater – a portmanteau of chocolate + rater… the choc equivalent of a sommelier (or for “oo la la” French-sounding sophisticrats try ‘Chocorateur’)
CQ – for Chocolate Quotient (a measure of baseline or center-point cocoa flavor free of any interference, nuances or under / overtones); a much-needed substitute for “chocolatey” / “chocolatiness”
Single-Estate – a lot of growers / brokers / manufacturers use ‘plantation’, a pejorative in the Americas with associations to slavery. For years we’ve encouraged “single-estate”, a baronial term but, hey, if Nancy Pelosi can brag about her $5 million Napa estate growing grapes, then why not chocolate?
Pod-Strength – the equivalent of scotch’s cask-strength or 100% unsweetened chocolate with no sugar added; just elements naturally found in the cacáo pod
The list keeps growing…
Perhaps some of these will stick in the vernacular. Hopefully our effort inspires the industry to codify an acceptable glossary.
The upshot: some complain as one person did that our reviews leave them “exhausted & uniformed” (a bit of a contradiction as well as a paradox, doubly so considering this person eagerly & often emails us for yet further info on our reviews listed on the Recent Reviews page) while others leave compliments that “This is the best article ever written about us & our chocolate. We learned from the article as well. You are a true journalist & critic. It is like a story, a book. Yes, I read your reviews as a great book! Your writing style – it is insane! I love it. Bless ya.”
“How long do you spend on a single chocolate review?”
No more than 2 hours for the actual write-up to “keep it fresh”.
Similar to music recording, a band can overdub a track to death or cut it in one take to give it more of a live, animated feel. A typical review usually takes an hour to write up. For very special ones that merit a more thorough evaluation & accounting, that can stretch to twice as long.
“How do you avoid bias or preconceptions in your reviews?”
Blind Tasting -- you’d be surprised how well this technique alone works. Once I gave a cocoa maven a Dark from a barsmith that this person absolutely adored & refused to ever say an untoward word about, even when their stuff was dog-product. During the blind taste test the maven tore into it & ripped on it unmercifully. Upon revealing the name & make, this individual could not believe it with their own eyes & was totally shocked that such sewage came out of their favorite maker.
the C-spot’s® general ethos – “no sacred cows” – largely takes care of any preconceptions.
As an independent consumer advocate, we hold no vested interest in any chocolate ventures. We rarely cozy up to others in the biz. This has its costs, both in terms of financial & social capital. It can be a lonely enterprise. And we’ve paid for it thru ostracism from an industry that pays consultants & bloggers to kiss up. One trivial example: a barsmith-clown offered us its latest release for free if we retracted a prior unfavorable review of one of its other bars. Our response: “you think we do this for free chocolate?”
Again, the design & construction of the the C-spot® review format builds-in objectivity. Deconstructing a chocolate into its constituent features mitigates subjectivity. Of key importance in this regard & unlike other reviewers, especially in the wine world, is that we never award any points whatsoever based on “Preference”, “Opinion” or other personalized judgments which often are weighted for 20% or even 30% of a total rating – an enormously wide berth for skewing a bar or a bottle toward subjective impressions. Such a category provides a dustbin for sweeping in all sorts of uncritical / unfiltered “je ne sais quois” (to use French expression) into the final score. It’s intellectually & professionally lazy.
Employment of baseline referents to calibrate the session. This serves a 2-fold purpose: a) a standard measure vis-à-vis the test sample & b) taste receptors grow dull with age (pssst, many well-known alpha-tasters are past their expiration date & wouldn’t know good chocolate if it bit them hard on the tongue; doubly so for the pros working for the candy giants who rarely sample fine-flavor chocolate to begin with; ergo a Callebaut manager calling a stick of cocoa butter disguised as 70% “dark” bar the “best chocolate I’ve ever had”). So pure cane sugar (for sweetness), quinine (for bitterness), citric acid (sourness), salt, MSG (umami) are on hand if need be to calibrate the palate. Of course, eventually e-noses will make all of this arcane if not obsolete.
Finally the mental discipline. This involves how to concentrate while tasting a bar in order to tease apart its flavor. It ain’t what most people think, or do for that matter, & one reason why many / most miss it (‘it’ being the ‘flavor notes’)… a topic TBD (To Be Discussed) at my upcoming presentation at the Northwest Chocolate Festival in Seattle. Be there to unlock the keys of this mind-melt meditation technique.
And now a word or some conceptual grounding on subjectivity:
With respect to flavor(s), subjectivity is an overblown hoax of mass deception. It apes the overarching geist of the times: relativity.
Funny though… in terms of sound, humans have more or less agreed on some tones being more pleasing than others (for example, a triad versus a tritone; the latter definitely has its place & purpose, especially valued in idiomatic placements). This emerged over centuries & pre-dates Bach’s well-tempered exercise in fixing the major / minor scales. Pythagoras well before him theorized certain mathematical principles that led to the harmonic overtone series & acoustic relationships.
Likewise, the Western canon, that archive under siege from the School of Relativists, generally elevates, say, Beethoven’s 9th above Ga-Ga’s Born This Way. Or to be more street, an ice-cream truck jingle over a police siren.
Driving the theme home farther, we reach consensus on vision as well. No one except the blind (in which case they’d be cited for another infraction) can talk their way out of a moving violation for running a traffic light with the excuse of ‘officer, a doppler-effect in my field of vision makes red appear green’. (Try it & see what happens.)
Ditto touch: while some may prefer thorns to roses, they represent a distinct minority.
These senses we generally agree on. But for some odd twitch when it comes to matters of taste, common ground gets tossed out the window & trashed with ‘it’s so subjective’ and ‘way too personal’.
Scientific research is gradually debunking myths surrounding subjectivity & flavor. For further insights, see the C-spot® Crushpad article On Matters of Good Taste.
The responsibility of any critic should be to check the “I” & to decouple the personal from the professional while realizing the inherent limitations thereof. This owes much to Martin Buber’s “I-thou” / “I-it” relationships rooted in Kantian critical theory / phenomenology, & aesthetics. Credit also John Dewey’s no-nonsense approach to rooting perception in experience outlined in his piece titled Art as Experience, later picked up by Rorty who sought to dissolve the objective/subjective divide, undermining analytical epistemology.
Not to be overlooked in all this (nor to sound pathological either) is the fact that chocolate, especially really high grade / high octane chocolate, speaks.
That’s right.
I know this sounds strange, like when you talk to God, it’s prayer; when God talks back, it’s all of a sudden bi-polar disorder. But chocolate does speak. In volumes & in tongues. (It is, after all, Theobroma cacao meaning ‘god-food’.) It might be hard to figure out exactly what a chocolate says but based on taste alone it sure as hell says something. It can talk to you like that. By simply plugging into the source & listening / tasting, the reviews literally almost write themselves, replete with their own dialed up rating, & any coded messages therein. It’s that personality, often related in the Impact Statement, communicating with you & engaging in a conversation. Put another way: a bar is a letter to the universe. Reviews are the replies back.
And so The Laws of Chocodynamics, particularly the 2nd & 3rd Laws – Trust Your Pal (palate, that is)… good is what you like, followed by There are standards & some chocolate is better than others; recognize quality even if you don’t like it. These address how one goes about evaluating & rendering judgment without sliding into a comfort zone where everything reduces to personal preferences of “I like it”.
For that very reason, many of my own personal favorites are not necessarily the “best” & vice-versa. Truthfully, some of the stuff I really like could never be considered top quality material, just as, say, Vosges & Godiva ain’t really premium chocolate (though they masquerade as such & charge preciously for it).
Knowing the difference & being able to maintain this key distinction guards against overly subjective viewpoints.
Scott of Dallas Food I don't review chocolate. I don't take notes or keep "scores" for personal use. If it's a new chocolate or one that I'm revisiting after some time away, I'll sit down with the bar (with an unfouled palate), start into it, and--rarely with any conscious effort at evaluation--react to it. Maybe that reaction is to spit the first bite into the trash can and drop the rest of the bar in after it. Maybe I finish a bite or two, then wrap up the rest to give to a coworker or one of my kids. If it's good, I'll finish the bar. If it's great, I'll finish the bar and make arrangements to buy more. That's the goal: finding bars that I want to return to again and again (and then doing so). I can articulate, after the fact, why I reacted to a chocolate the way I did; but the root of the evaluation is always instinctual (though with instinct conditioned by a basic understanding of cacao and craftsmanship and by years of palate habituation with large quantities of extraordinary chocolate).
Chloe Doutre-Roussel I follow the basic methodology most teach and I teach, to listen to each of our senses one by one, ending with the aromas in the mouth/nose. Nothing secret, just making sure my body & the environment are in the right conditions, and listening the same chocolate, many times, each time focusing on a different sense and sub part of the sense (ex in the mouth you feel texture, flavors then aromas)- Unconsciously, automatically, my brain connects these” feelings “ with other bars I have tasted, adding this new chocolate in a specific place in my sensorial data base in my brain.
I hardly take any notes, unless I have to discuss or share with someone. By the time I have spent so much time, with so much attention, on a chocolate, I know it by heart, I think I could recognize it blind – If there is anything left over I put it in one of my wine cellars properly wrapped in clear film and add this new entry in an excel sheet of my “stock”.
I must admit I do not look much at other reviews, I know that words are often not enough to describe what one feels and that for chocolate, there is no education nor standards as there is in the wine world, even the coffee world, so often what other people write about a specific chocolate is different from what I write, or “think” as I do not often write. I prefer one to one discussions with people, tasting if possible at the same time the same product (same batch in same conditions of environment and same conditions of stoagre of that chocolate)
I make sure I do not read the ingredient list or too much of the packaging (besides the % of cacao that gives me an indication of the sweetness level) -I guess we can say I can not be biased often…
George Gensler When reviewing chocolate, I usually try the chocolate several times and at different times of the day. The first taste is to get a general overall idea about the chocolate. A day or so later, I do a more formal review, reviewing the package, the aroma, appearance, texture, and taste of the chocolate. I use hot water to clear my palate between bites and take notes throughout the tasting. This second part of the process can take several hours and about 4 squares of the chocolate, more, though, if it’s a particularly complex chocolate. Several days later, I retaste the chocolate with my written review in front of me, making updates, if necessary. The total amount of time spent on a bar varies, but probably ranges from 2-4 hours all told.
I don’t read reviews of bars that I’m planning to taste until after I’ve tasted it myself. I don’t want to influence my own experience of the chocolate. In addition to not reading other reviews before I’ve tasted a bar, I also try to avoid bias or preconceptions, by not reading tasting notes on the package or the ingredient list. Whatever preconceptions I may have about a chocolate maker, I taste each bar on its own merits. The “same bar” can taste different from harvest to harvest so I always give the manufacturer the benefit of the doubt. Even when a maker has a particular profile, I try not to anticipate those flavors in the bar I’m tasting.
Maricel Presilla I look at the color and look of the bar. I examine the bar for defects. I taste it once for an overall impression then I go back to it very deliberately, never chewing, letting it melt. I let all my senses become engaged in the experience of smelling and tasting and I analyze my sensations every step of the way. For instance, I analyze the length of the experience, how the bar feels every second as it melts in my mouth. I ask myself, “How is this chocolate evolving in my mouth? What comes first? What comes second? What is the lingering flavor, how long does the experience last?" For me it is an overall experience. I look at every detail.
Alex Rast Definitely. I have a standard review process. As you may see from SeventyPercent.com for formal reviews there is a system in place which makes it easy to proceed through the technical parts of the review. A different but similar-in-concept system is used in competitions. For a "full formal" review I always schedule to do these right after climbing; this is when my taste sensitivity reaches a maximum. I try to use an ideal of 50 g for a sample; less and one doesn't capture all the notes in the length, more is just...more. Before actually eating I smell it for a good minute or more (with a possible time to allow outgassing if the chocolate was in a plastic wrap) I will start with the largest bite I can practicably make, and chew it quite completely without swallowing to get best chocolate distribution. This is critical because the initial taste conveys more of the flavour than every other stage, not that there shouldn't be other flavours but things you miss in the initial taste you'll *never* detect. Subsequent bites are slower and more thoughtful. I reserve the last bite to evaluate the texture, completely blanking out the flavour so that I can focus on the melt. I also always write up the tasting notes immediately, so that nothing is forgotten.
As for what's different from other reviews, the one part I consistently notice is in identifying distinctly those components that are part of the aroma from those that are part of the flavour. I notice quite a few reviews that note as flavours things that are actually aromas. In truth, the two are tightly coupled, so it's not quite as discrete as that, but I do think being clear on aroma versus flavour also permits clarity on potential for quality as opposed to actual quality.
Avoiding bias is hard for any judge, particularly with manufacturers that have a known reputation. I'm not sure I, or anyone else, is qualified to state that they are unbiassed; this is a characteristic that can really only be identified by an external third party. By yourself, you can't really conceal or randomise samples, the normal ways of eliminating bias. You can, however, identify bias over the long term of many chocolates tried, using statistical methods. That's not something that can be quickly explained but the gist of it is, by looking at various reviews, evaluations for a given brand or source that consistently stand out from the statistical "pattern" probably suggest an internal bias of some sort.
Richard Vaughan
Great questions. Before beginning formal reviews, I spent several weeks identifying the qualities of chocolate that other reviewers were using, thought about what additional information I would want, and put together a spreadsheet in which to capture the data. Over time we discovered that some of our quality dimensions weren't distributing well (such as length), so we re-scaled them to better distribute the results. Also, we found our initial overall rating scheme of one to four stars was too general, and after soliciting feedback from several other chocolate experts, we revised it to a scale from 0 to 10 with 0.5 increments. The data we track include: chocolate maker, bar name, region where cacao was grown (if not a blend), specific plantation (if any), the "type" of cacao (Criollo, Trinitario, Forastero -- please see my answer to #6 for more about this), year the chocolate was made, best before date, size of the bar, cocoa solids %, added fat (if any), added sweetener (if any), added emulsifier (if any), added flavoring (if any), other ingredients (if any), list price, color (using PANTONE), texture (smoothness, sheen, scoring, bubbling, etc.), aroma, snap, taste, melt, length, and finish.
We photograph the wrapper ahead of time. At our initial evaluation session, we begin by photographing the chocolate bar. After examining the appearance, texture, and snap, we then make two or three passes and take notes on the aroma, taste, melt, length, and finish in that first session. We might taste multiple bars in an initial evaluation session, using bread and water to clear our palates between passes and bars. To help ensure consistent data by hopefully eliminating situational effects (for example, anything we might have recently eaten), I evaluate the chocolate again on at least one other occasion before posting our results. I like to do that session in the morning a few hours after brushing my teeth without toothpaste and not having consumed anything other than water. At some point, we edit the photos, cropping and adjusting the color to make them as close as we can to actual wrapper and bar appearance (Lindy is much better at this than I am, so the good photos are usually hers). While writing the review for the blog, I make a final pass with the chocolate to confirm our earlier results. Finally, I search the web to find the lowest prices for purchasing that chocolate bar online, and include links to those for each shipping area (typically one for North America and one for Europe). Overall, it probably takes about 3 to 4 hours for each review.
When we first began reviewing chocolate, it was easy to avoid preconceptions, as we didn’t know what to anticipate. It now takes a little conscious effort to be completely open to fully experiencing each chocolate, and not, for example, to anticipate certain flavors from single origin chocolates based on typical regional characteristics. We try very hard to evaluate each chocolate independently on its own merit.
Ian Whitaker Yes, standardisation is necessary to obtain the most reliable results. I think that our evaluation process will be best explained by answering your subsequent questions in depth.
The World Chocolate Awards are completely impartial and independent. The World Chocolate Awards was founded because, for the fine chocolate industry, no mark of distinction or award conforms to these standards. Everything I am about to relate below is far from the norm. So we have something valuable, new and unique to offer our readers.
To ensure that we are impartial and independent, no one who works for the World Chocolate Awards can have worked for a chocolatier in any role. They must have never worked in public relations for, marketing for, or undertaken any kind of promotion of the chocolate industry. Neither can they be related to a chocolatier by family.
The World Chocolate Awards is also unique, impartial and independent because it does not accept payments from the chocolate industry. Also, we do not accept gifts of any kind that could be viewed as a substitute for payments. We do not accept free bars of chocolate or any other products that we evaluate.
Not only is the chocolate that we evaluate paid for, but we also avoid the possibility of chocolate bars being specifically produced or selected by a chocolatier to be judged. We ensure that we evaluate the same chocolate bars that the public buy.
The World Chocolate Awards has standards designed to avoid bias in many other ways: by storing chocolate correctly; ensuring a clean palate; consistent tasting conditions; consistent expectations; and by evaluating a chocolate on a number of occasions.
Evaluating a chocolate on more than one occasion allows for any possible variation in the taster’s ability to perceive flavour from day to day. It also allows the taster to become accustomed to new flavours with neither the excitement of novelty, nor the disappointment of unfamiliarity. It allows the taster to gain a sufficient understanding of a chocolate’s flavours before making a conclusive evaluation.
This method avoids the pressure and the impracticalities with the method commonly used in chocolate awards and competitions: tasting over a hundred chocolates in one day, tasting a very high number of different chocolates, each supplied in a very limited quantity, in a very limited time, in very unusual surroundings and circumstances.
