Judges for this competition are selected based on their reputation, their professionalism and their extensive experience cupping teas (minimum of 5 years of cupping teas for purchase, quality control, development, etc.). Each championship will have a minimum of 3 with a maximum of 5 judges. Judges are not allowed to judge more than 6 consecutive championships. At which time, the judge must wait at least 2 championships before judging again. However, there will be at least 2 Judges, on the panel, that have prior experience judging this competition. These regulations do not apply to the head judge. Judges are not paid, but are reimbursed for their travel expenses. Furthermore, judges will have their head shot and biography marketed on the championship website or other World Tea Media marketing materials, like the annual buyer’s guide.
Each championship will have a Head Judge that will work with World Tea Media before, during and after the championship to ensure efficacy of the championship. This Head Judge is charge with managing all other judges and support the championship in whole. The Head Judge must follow all other rules and regulations as it pertains to being a judge.
Judges cannot be directly employed by companies that have entered this championship. However, it is conceivable that a judge may be indirectly involved with companies that have entered this championship either by supplying the company ingredients, selling them finished blended/flavored products, have been previously employed by the entrant, or by consulting with them. Although we know the integrity of these judges will not impair the ability for a fair and just evaluation, we recognize the need for additional measures and controls to ensure that there is no potential for the results to be affected by any judge who has an interest in a particular tea entered into the competition.
The simplest means to accomplish this would be to not have a judge evaluate the tea; however, having one judge’s rating removed from the average but not the others would potentially skew the overall results. Therefore, we are using the following measures and controls:
Prior to the Competition – Each of the judges will be required to sign an agreement to maintain the highest ethical standards during the competition and are further required to disclose any direct or indirect involvement to a particular company that has entered. Judges who fail to uphold these standards will be immediately dismissed from judging and will not be allowed to judge again.
Initial Evaluation and Rating Process – All teas shown to judges are internally coded so that the judges have no direct knowledge of the tea that is being judged. This ensures that the teas are judged blind. For judges who have a specific interest in an entrant, their evaluation of the tea they have an interest in will be statistically evaluated against the other judges’ evaluation of that tea and in comparison, to all others teas in that category. Any evaluation from a judge that had an interest in a specific tea that has more than two standard deviations from the average will result in discarding that specific judge’s evaluation for the entire category. The comprehensive methodology was created by UNLV Professors who are experts in statistical analysis. (See statistical methodology for eliminating bias and errors section for more details.)