Aggregate scores in cupping notes

[jameso]

I've been wondering about this for a while, and Steve's new information website jogged my memory to ask the question.

What is the value of the aggregate score in cupping notes?

Some of the individual scores I can appreciate are a reflection of a coffee's quality - things like finish.

But others, like acidity, are surely different. I'm interested to know how much acidity you can taste in a coffee. But whether a high acidity should contribute to a high score is a different question. In the case of a Kenyan, yes. In the case of a Brazil, no. (I like Cachoeira, but it is definitely not a high-acidity coffee. Why should its score suffer for this?)

Or am I a newbie who doesn't understand how something like "acidity" is scored? Is it about whether the acidity is appropriate rather than how much? If so, how do you pick a number?!

(Incidentally, I notice that on the coffeearticles website, Gethumbwini and Cachoeira both score 7 out of 8 on acidity. I must be missing something somewhere. To my palate, the Geth has a much higher acidity than the Cach. Please help!)

James

[kingseven]

With acidity on most cupping forms there are two sections. Not only is the strength of the acidity rated, but also how positive or negative that acidity is - fresh/clean/pleasing acidity versus sourness.

Roast is obviously key to percieved acidity, hence when cupping great pains are taken to make sure roasts are identical across all coffees. I'm not sure how the Geth and the Cach were roasted when you tried them.

Lets not get into the effects of brewing just yet.

Are the CoE scoring sheets available online - they might help it make some sense...

[jameso]

I was looking at www.coffeearticles.co.uk. I don't know how the cupping forms on there compare to the CoE ones. They are different from the Sweet Marias ones, which is what I've seen before.

When I tried the Geth and Cach, both were roasted in the Hottop to the point when I was sure I could hear 2nd crack. I.e. very definitely pops, that aren't left-over 1st cracks, but not rolling either. Geth had a pleasant grapefruity acidity; Cach was much mellower with caramel.

Does anyone have the CoE scoring sheets - I'm interested in seeing what's on them...

James

[Beanie]

http://www.coffeeinfo.co.uk/index.php?n ... load&cid=3

BTW, nice one Steve Nice on the cupping notes and soooo glad with the descriptions of recommended roasts

[ubo]

jameso - wouldn't get too hung up on CofE or similar type forms, they don't stand up to much statistical scrutiny and are as much an aide memoir as anything else, but too an extent that (perhaps slightly cynical ) view is missing the point as what they do provide is a relatively simplistic way to denote sensory quality using terms that one hopes are easily understood by international judging panels of variable ability and views of the 'quality requirements' of the origins that they are tasting. One cupper's fruit is another's ferment is often a bone of contention for example. In an ideal world you would use far greater vocabulary but that would be a lot more complicated and time consuming to deploy and way beyond the resources of the CofE (who do a great job as far as I am concerned).

[jameso]

Thanks B_B for the downloads link. Helpful

Thanks Ubo for the reminder that they are only a tool. Cynical. Moi?

Looking at the various cupping forms on the coffeeinfo downloads site, the only quality that isn't self-evidently one for which "more is better" is acidity. So the answer there is probably Jim's one: You give a coffee a high score for acidity if it's acidity is pleasant.

Why do I ask? I want to understand cupping sheets that others have read. Getting there...

[Steve]

A high acidity score doesnt mean that there is lots of acidity just that the acidity suited the other parts of the cup, its one part that makes the whole cup hang together.

Ubo I'm interested in developing the thought of how something more extensive could be implimented, not for COE but more for personal reasons. Wha textra do you think would be valuable?

Steve

[ubo]

Steve

There has been only limited formal work done on development of vocabularies for coffee (versus those for other products such as whiskey and beer which have a longer history in terms of sensory analysis, certainly in the literature). The ICO have carried out some studies in the past (e.g. their work on a vocabulary for East African coffees dating back to the early 80’s, author was Hal Macfie off the top of my head) but most of this has been defect based and I think the industry and the market is now realising the value in an approach which looks to better define a coffee from a positive standpoint or one that allows comparison on non-defect attributes to in some way allow comparison from a qualitative standpoint as per what CofE is trying to achieve.

Enough rambling, there are a couple of ways to develop a vocabulary and it depends on what you are looking for but for a start point:

1. Is it to be consumer based? i.e. should it have relevance to consumers by that I mean are the terms user friendly and will it be used to review coffees in consumer format, e.g. brewed as filter, espresso, latte given the fact that cupping extractions have artefacts and profiles that are somewhat alien to certainly the UK consumer.

2. Is it to be industry based? i.e. probable use of some defect vocabulary and terms such as acidity (which don’t translate to most consumers)

To illicit a consumer based vocabulary one way to go about it is to have consumers into structured focus groups present them with drinks in the format for which you want to develop the vocabulary and then basically look and see what you get. The important element here is the structure to the management of the group, for instance while being careful not to lead them you could ask the consumers the relevance of certain terms in the description of products which you may find more commonly in more technical vocabularies. You can also run a free choice profiling experiment where you allow consumers to use their own vocabulary to describe products; you can then actually derive quantitative data from this using a statistical tool called generalised procrustes analysis (developed in the 70’s by Gower). The latter is not for the faint hearted or numerically challenged, requires dedicated statistical software and ultimately is perhaps of limited value in terms of true vocabulary development. What would you expect to get out of all of this, perhaps 20 -25 vocabulary terms after you have distilled them down.

On the industry based side, while I would question the relevance of such a vocabulary (aside from one cupper talking to another cupper, but then again they would never agree to use the same terms!) you could simply pull together references from various cupping forms / vocabularies around the world. I once saw Ken Davids present a vocabulary at the SCAA of > 100 terms and suggest that they could be used quantitatively, now you could never use these statistically as it would be impossible to train to use so many terms consistently. Again you would have the choice as to whether to include defect terms or not depending on what you planned to use the vocabulary for.

Having carried out either of the above, at this point you could just stop, in that you would have generated terms to describe your coffee and for some that would be enough. To actually start sticking numbers against these to get quantitative values from cupping or sensory sessions that are actually statistically relevant and meaningful is a whole different ball game and would require as a minimum the following:

- panellist / cupper selection, e.g. screening for anosmia on relevant compounds and sensitivity test as well discriminatory test using triangle tests (as a guide for statistical relevance you would need typically 6 - 8 good panellist / cupper on every session)

- training, training and more training on vocabulary terms using training compounds and actual samples with statistical analysis and interpretation of panellist / cupper performance

- experimental design to ensure no session effects (e.g. randomised sample presentation using latin squares)

- ability to statistically analyse and interpret results from sessions (from a roasters standpoint that could mean also correlating process data or blend data to sensory data) for comparison of products. There are a bunch of sensory statistical methods you can use to normalise panellist data (if required), identify vocabulary terms where the variance is too high and therefore they are significant in that particular experiment etc, etc

Seeing the detail, effort and time that goes into getting truly relevant statistical data this does make me slightly sceptical of the scoring / quantitative systems employed by CofE and similar programmes in a number of areas but I still don’t think that detracts from the fact that they do highlight good coffees and quality within the industry. However to date I don’t think they have been overly successful in reaching consumers but I guess this will come in time.

Lastly (and perhaps I should have put this at the start ) to answer your question more succinctly, you might want to check out: Guidelines for Sensory Analysis in Food Product Development and Quality Control, (Carpenter, Lyon & Hasdell, Aspen, ISBN – 0-8342-1642-6). This is a pretty good intro text without getting too much into the statistical requirements.

[Steve]

I'm hearing you (and agree with most of what you say)but how important do you think the vocabularies are to the average whiskey drinker and wine drinker. Now to me its important but thats because I love the whole experience, but most of my friends just roll there eyes and take another gulp.

I feel coffee is a bit like this, but there is no need to dumb down for the consumer, (I certainly dont see the wine market or the malt market doing this|). Give them the info and if they want it they will find it, if they don't then it will come.

1. Is it to be consumer based? i.e. should it have relevance to consumers by that I mean are the terms user friendly and will it be used to review coffees in consumer format, e.g. brewed as filter, espresso, latte given the fact that cupping extractions have artefacts and profiles that are somewhat alien to certainly the UK consumer.

But inst it more important to raise awareness and knowledge or at the very least quality from the status quo ?

2. Is it to be industry based? i.e. probable use of some defect vocabulary and terms such as acidity (which don’t translate to most consumers)

Acidity a defect? Inst this why there is ferment? I agree its the hardest understood by the consumer but I feel central to understanding the make up of a coffee taste. I wish there was something more descriptive to use but I've spent a long time thinking about it.

Should defects be finding there way to a roasters cupping table (or competition table)? Yes I know they do but is there a real need for this to have added vocabulary for this? Surely rejected is just fine. I think this is more important at origin or with the importers working with farmers to make them understand what the market wants.

However to date I don’t think they have been overly successful in reaching consumers but I guess this will come in time.

I think thats coming. We must remember in real terms its a young program that has had to grow up very quickly. If you had asked 2 years ago what is COE most people would not have known, now I think if you are slightly into coffee you would and on these boards its up in the headlines.

I do wonder how important it is to reach consumers. As long as coffee improves in quality and equality (every one getting a good deal) then thats whats really important.

Lastly (and perhaps I should have put this at the start Embarassed ) to answer your question more succinctly, you might want to check out: Guidelines for Sensory Analysis in Food Product Development and Quality Control, (Carpenter, Lyon & Hasdell, Aspen, ISBN – 0-8342-1642-6). This is a pretty good intro text without getting too much into the statistical requirements.

More reading material Will take a look at this always looking to improve my knowledge with new stuff.

I love threads like this

Steve

[ubo]

Not great at quoting quote threads so apologies for my clumsy response:

Ref, wine / malt - likewise, agree with you here but I think where wine and whiskey have been successful in doing is leveraging consumer based vocabularies (which I wouldn’t necessarily viewed as dumbed down) in product development and promotion. More power to the coffee industry if we are to do this as well.

Ref the status quo / raising awaress - completely, but too many roasters still start and finish their product development on the cupping table as opposed to making it a jumping off point.

Ref acidity a defect - semantics possibly here but I said ‘defect vocabulary and terms such as acidity’ meaning in the context of the sentence that positive terms such as acidity can be difficult to understand as you also agree with – I wasn’t suggesting that acidity is a defect although its balance and character are complex by nature, e.g. high acidity in Sumatras is a bad thing but Japanese buyers often go for fully-washed Sum's which have heightened acidity. I think this is also where other terms are useful which explore e.g. different types of fruit acids for instance.

Ref defect vocab for cupping table - sure but as the good work carried out by people like Mercanta and Wakefields in the UK brings producers closer to roasters then I think it is important that roasters can identify and verbalise defects. It is also crucial in building roasters understanding of green production and quality which is ultimately surely well aligned with programmes like CofE. I think what is useful is to identify terms as ‘defect terms’ if you know what I mean.

Ref CofE - I would agree that there is a drip down effect from a quality standpoint and there is perhaps an argument with regards to the buyers as being the’ usual suspects’, albeit that these roasters are often at the vanguard of coffee quality. I don’t think that CofE ensures equality particularly well, internally given the current judging system and from a wider aspect I question given its current scale how much impact it can have in raising returns for farmers versus schemes like SBUX’s Café Practices and Fairtrade particularly given the lack of consumer awareness / understanding of the scheme.

Njoyin this thread 2

[Steve]

Ref CofE - I would agree that there is a drip down effect from a quality standpoint and there is perhaps an argument with regards to the buyers as being the’ usual suspects’, albeit that these roasters are often at the vanguard of coffee quality. I don’t think that CofE ensures equality particularly well, internally given the current judging system and from a wider aspect I question given its current scale how much impact it can have in raising returns for farmers versus schemes like SBUX’s Café Practices and Fairtrade particularly given the lack of consumer awareness / understanding of the scheme.

I agree with it most of what you say apart from the impact the COE has and the judging system. I cant think of anything else that get a diverse group of people who may otherwise not go to origin to meet farmers and experience the culture of a country. It completely change the way I viewed coffee and the people who grow it. Also indulging a whole week in one countries coffee made me really begin to understand what that country can give you in the cup.

A diverse jury means that there are some people who's cupping skills are better than others. I must admit at the first competition I felt quite over whelmed with the vocabulary and the names on show. But there is no better way to learn than cupping with some of the big cuppers for whom I have enormous respect. This grows everyone's cupping skills so increases the green buyers knowledge.

The effect for the farmer is he can begin to understand what the buyers want, raise there self esteem and belief they can work hard and be rewarded for it. Also it can crate long term relationships with roasters that can last for many years, and raise a farms profile within the community.

I cant think of any better way, that has such universal support.

Steve

[kingseven]

Just as a side note - I am unaware of any common anosmias relevant to coffee, by which I don't mean there aren't any - but I'd like to know more.

Their relevance is also very important. I am have a milder reaction to prop than most people (though I can taste it), but I've yet to find that has any bearing on my ability to taste coffee.

I think suppression/release stuff is very interesting when it comes to cupping, and it is something I am very aware of when I cup (especially when smelling the dry)

Shame, mr ubo, that we couldn't meet when I was in your neck of the woods. I gather you were off shopping....

[leecb]

I have to say I agree with Ubo about the CofE's place in the wider coffee market. It remains, in my experience, very low in profile compared to other schemes. This does seem a shame as most of the other schemes don't seem to be about quality of the final product (which while of course the price the farmer gets for his product is very important, is not the main concern of most end users). CofE needs some serious profile raising outside the very crema (sic) of the industry to have a real impact on the quality of coffee.

It will always be for a very small group of, as Ubo says "the usual suspects", but to the serious big players seems of questionable relevance or interest, and to be of use to the end user needs to be a powerful forum for quality in the whole market eg a driving force in raising standards of coffee across the board, in all areas that grow coffee.
Surely this is the opportunity it should be grabbing and running with?

Highlighting the very high quality of a few Central/South American plantations coffees seems to be only part of what is needed in the growth of quality. Surely a drive to improve best practice, in all spheres of coffee production, all around the world is what is needed to improve the quality of coffee in any meaningful sense.

As for the Starbucks scheme, it is hard to imagine it being anything other than a fig-leaf for conscience! And whether Starbucks end users have any interest in what is in their gallon bucket is a moot point!!

[Steve]

I have to say I agree with Ubo about the CofE's place in the wider coffee market. It remains, in my experience, very low in profile compared to other schemes.

I disagree. The fact we are talking about a program that sells lots as small as 700kg proves its profile is very high. The comp gives the small roaster a chance to make a statement about the coffee they buy, and more importantly show the consumer how good coffee can be. There are lots that aren't COE that are very very good, but this rewards the farmer financially. Its shows thats its been through some process to get where it is.

I also think it is the usual suspects that really want to do something about marrying quality to an increased price. And that increased price dwarfs that of fairtrade etc. I think this is shown by auctions like the Q auction that sell large lots in a similar way to to the COE but gets prices no where near. The "big boys" don't want to pay a penny more than the consumer thinks they should.

The fairtrade brand in coffee is flawed, and the only reason its being used by the larger roasters (in my opinion) is because average joe public thinks its great, its the easy way out. A few weeks ago the New York price went above the fair trade price, and the fair trade farmers got paid no more, is that fair? There are a lot of people who have become very successful and wealthy of the fair trade scheme but I dont think to many of them are farmers.

It will always be for a very small group of, as Ubo says "the usual suspects", but to the serious big players seems of questionable relevance or interest, and to be of use to the end user needs to be a powerful forum for quality in the whole market

The "serious big players" don't care for a number of reasons, one the lots aren't big enough for them first of all (but the Q auction where they are big enough they don't buy big) and I also think they think the consumer wont get it, and for most people I guess thats probably right. Specialty consumers do, but most people wont. By making the lots smaller they drive up the price for the farmer and give it access to more people.

Boutique lots is often used to in a derisory way towards the COE but I think it should be embraced for what it is. The cream of the crop from a hard working farmer. This program for all its faults is by far the best we have.

Surely a drive to improve best practice, in all spheres of coffee production, all around the world is what is needed to improve the quality of coffee in any meaningful sense.

Improving the quality in a meaningful sense I think is a bit harsh. What would suggest is done? Try telling the winning farmer from a competition that it isn't meaningful, because the two winners I met were incredibly proud of there achievements.

I don't see anything else waiting in the wings to pick up the baton. Its a start, that is making a difference and will continue too.

Hey this is a young program with plans to expand, but it needs to build good foundations and this is being done in south America as thats where the forward thinking specialty associations are. The coffee markets of every country is complicated, and has its own set of rules. Many of them closed shops. This program is here to stay, and will grow, and at some point the "big boys" will have to take notice.



Steve

(still loving this thread )

[leecb]

Steve,
I don't disagree with anything you say.
Of course a program that rewards farmers for their attempts/successes at raising standards is absolutely right,
And I wouldn't suggest that it holds no importance for the winning farmer! I imagine it is a huge incentive for all the local farmers to raise the standard and as long as this is run hand in hand with the need to raise the standard of practice in all areas, is absolutely right.

My comments were about the relevance of this scheme in raising standards when it remains so small.(Obviously those who benefit from it not included.)
I hope that it does indeed expand and become an International Enterprise less rooted in the Americas so that others can benefit from it.
But first and foremost any operation like this should in my opinion also be about teaching as many of the end consumers as possible that there is something better out there than the ubiquitous Starbucks bucket of swill, and to do this a wider audience must be reached. Obviously not all consumers are going to be interested but since when should we address efforts at the lowest common denominator? As long as consumers are happy to accept that bucket of swill, the big players maintain their power!

These efforts are always driven top down and CofE has a lot to offer, but to show people that what they are getting is unacceptable you need to reach a wider audience than is happening at present, and to be honest America is not often seen as an arbiter of taste, they did create these super players after all!

I don't pretend to have the answers, but think this is a really important discussion!
I so rarely buy coffee these days as am always disappointed by the outcome and to me that seems a real shame. We must try to raise the standards of what people receive for their not inconsiderable 2.20 a cup (in my neck of the woods!) <i>And I'm sure the farmer receives a negligible amount of that!</i>
Coffee pricing is also subject to the distortions created by futures market trading and hedging (a hugely complicated issue!) which has no interest in a stable market place, or the interests of the farmer.

I'm sure that many businesses who "support" the Fairtrade scheme also have no interest in the farmer, but while I'm sure it isn't always successful, it does seem to have
A. raised the awareness of the consumer to the problems for the producer in the way the market functions and
B. improved the price many farmers receive for their product. (And it is well known!)

Very much interested in this thread too, and enjoying it three!!!