2021 Post-Competition FAQ

By John Moorhead, National Homebrew Competition Director

We are extremely proud to have been able to hold the National Homebrew Competition (NHC) amidst COVID-19 restrictions on gatherings, and we could not have done it without the support and dedication of all our judges, competition staff, and volunteers.

We know we changed a lot of things to make NHC happen this year. In the spirit of continuous improvement, I’d like to address a few of the larger themes I’ve heard about from entrants surrounding the competition.

How was this year’s competition format different from previous years’?

Each category in the 2021 NHC was divided among multiple judge panels—think of each category as its own bracket. Each judging panel evaluated a flight of entries within a category at one time; all entries in that flight were presented at once. Each panel evaluated the entries in front of them, discussed the flight, and advanced three entries to the next round until a final round of no more than 12 entries existed. This format was much different than queued judging, in which a panel evaluates and discusses one entry at a time.

The final round of each category was judged by a panel of four judges who determined first, second, and third place. First-place entries in each category moved on to the best-of-show round to determine the major awards for beer, mead, and cider.

Throughout the process, judges only saw entries in sample cups, never in original containers, and never with any identifying information about the brewer or entry (e.g., brewer name, entry name, email address, etc.). The entries were prepared in a staging room by highly trained volunteers to ensure quality control.

Why did the Rules & Regulations change? 

The Rules & Regulations are released in the months leading up to the competition. While we work hard to get everything as close to concrete as possible as early as possible, some things do change. With a year of new processes, procedures, and format, part of competition planning is to update information. 

A group of 20 experienced judges tested comment sheets with scores and comment sheets without scores. Discussions with those judges revealed that eliminating the scoring process would save time while still giving every beer a fair evaluation and a fair shot at moving to the next round. Such time-saving methods were necessary to complete judging on time without running 14-hour days.

The base style is in the entry name. Why wasn’t that shown? 

The National Homebrew Competition is a double-blind competition, which means entries are renumbered after receiving and prior to judging, and all identifying information and markers are removed. Bottles are handled in a staging room, and entry names are never displayed. We cannot assume an entry’s base style based on a name.

The Rules & Regulations state which entries require a base style per the 2015 BJCP Style Guidelines (entry instructions may be found in the 2015 BJCP Style Guideline based on the subcategory). Identifying information found in the Specialty Information section is removed to maintain blind judging and the integrity of the competition. For example, if you use an identifying marker such as a geographical term or brand, that is removed. This is not a new rule and has been a policy of the competition for over a decade.

Why do comment sheets only have a Judge ID?

The primary motivation for anonymity is to provide judges a comfortable space to share impressions of an entry, consistent with the competition’s blind nature. We believe this process provides more unvarnished feedback and further enhances competition integrity.

The judge pool is audited every year to ensure quality comment sheets. It is challenging to ensure that each judge provides adequate feedback until after the competition is complete—the system is not perfect, but it continuously improves and evolves.

Judges are selected based on a combination of formal sensory training, experience judging in competitions, positive judging demeanor, advanced knowledge of beer styles and brewing process, and peer recognition.

Remember that all judges are different, as are all drinkers and homebrewers, and it is unrealistic to expect uniform sensory evaluations from a judge panel in a competition.

Here is a list of the 2001 National Homebrew Competition judges.

Do we receive certificates based on how an entry advanced?

Unfortunately, we will not be able to deliver certificates in 2021. In the past, we have had certificates based on scores.

We had originally planned to provide certificates based on how an entry advanced in the competition. As we migrated the competition software to a new system and worked to put on the competition, we prioritized competition software functionality and reporting to ensure data integrity and accuracy, followed by emailing out comment sheets. We will look into an effective way to provide certificates in the coming years and apologize for not having them this year. 

Should I have two comment sheets for each round?

No. There are comment sheets for the first round only. As in other competitions, judge panels evaluate entries in the first round, advance entries, and perform mini-best-of-shows in subsequent rounds until a final round of a category is reached.

This is the same as in years past. The difference is that the 2021 competition was a single-site location, while NHC has historically used a two-site model. With no need for entrants to rebrew and reship after advancing from a first site to a second, we did not re-evaluate entries after the first round of judging.

Why did the feedback sheets remove scoring?

We did not communicate the changes effectively enough, and we did not notify entrants early enough. We recognize that the timing of the announcement of the change and the removal of numerical scoring have left some entrants with a sense of violated expectations. We regret not doing better in our communications. 

We ultimately made a group decision to remove the scoring component in the interests of executing the competition successfully and accommodating as many entries as possible. We felt that the disappointment of not being able to host the 2020 National Homebrew Competition made this especially important.

Moving away from queued judging changes the function of a score. In queued judging, one judge may assign a high score while another assigns a low score, and a consensus is found prior to moving on to the next entry. Scoring in these situations is useful because the number offers the basis of consensus; however, a score doesn’t necessarily indicate whether an entry advances or not. 

Judge panels evaluating an entire flight find consensus through notes and discussions, by which they advance the top three entries to the next round. In this format, scores, which are assigned prior to discussion and not used to form consensus, carry less meaning. After discussion, an entry one judge scored highly could be excluded, just as an entry another judge scored less highly could still advance. In a flight-based competition model, discussion and deliberation form the basis of consensus, not a numerical score.

Discussions of the best evaluation model for 2021 NHC occurred after the competition opened. We made changes to the comment sheet after many conversations and timing tests with our competition operations team and with members of the AHA competition subcommittee. The change grew out of a concern that we would not be able to offer sufficient time to each entry using the previous document, given the time available for judging.

Understanding that it was an impactful change and remaining committed to hosting the competition, we announced the new comment sheets prior to judging. We communicated the changes in an email to all registered entrants and published an announcement on the competition website in mid-April, along with attachments of the new comment sheets. We updated the rules and regulations to reflect the changes.

Final thoughts

We are proud of the 2021 National Homebrew Competition and our ability to hold an incredible event despite the challenges presented by the COVID-19 pandemic.

As we do every year, we are already looking at ways to improve the 2022 competition and make it an even better experience for our members. We look forward to sharing those improvements with you all later this year.

The post 2021 Post-Competition FAQ appeared first on American Homebrewers Association.