Google changes (and simplifies) its Search Quality Raters Guidelines

A minor update, with changes that do not disrupt the substance of the information and guidance but are intended to simplify some aspects that might have been more difficult, unclear or outdated. As of last November 16, Google published the new version of the Search Quality Raters Guidelines, which are the guidelines that serve as a compass for the evaluation work of its quality raters.

Guidelines for Google Search Quality Raters, the November 2023 update

Announcing the change is directly in a Search Central blog post by Elizabeth Tucker of the Google Search Quality team, who immediately points out that the update is intended to “simplify our guidelines.”

Specifically, Tucker explains that the intervention affected the NeedsMet scale definitions (which measure the satisfaction of user needs), with additional guidance added for different types of Web pages and modern examples, such as the inclusion of newer content formats (particularly shorts, short-form videos), the removal of outdated and redundant examples, and the expansion of the classification guidance for forums and discussion pages.

Google also adds one relevant piece of information: none of these involve major or fundamental changes in the guidelines.

What specifically changes with the new version

The previous version of the guidelines dated back to December 2022 and was (that one) particularly relevant and interesting because it officially added the E of Experience to the concept of EAT, thus paving the way for the new EEAT paradigm.

That document consisted of 176 pages, while the current one stops at 168 pages: the first novelty, then, lies in the quantitative reduction of information and probably ties in with the simplification work cited by Tucker to help quality raters.

Recall, also through the words of Search Engine Land, that Google quality raters are “used by external consultants hired by Google to assess the quality of Google search results”: they do not directly influence search results and cannot downgrade or upgrade a specific site or page in Google Search.

As Barry Schwartz confirms, “at first glance, it appears that this was not a significant update”; in particular, the expert noted some minor changes “in sections 9.2 Ratings for Forums and Q&A Pages, section 12.1 Important Rating Definitions and Ideas, 12.7.4 Visit-in-Person Queries and User Location and the Part 3: Needs Met Rating Guideline section,” but nothing so important that it needs to be thoroughly investigated.

Interestingly, however, Google also promptly updated the overview summary document on the guidelines, changing some screens and definitions again in the Needs Met section.

Specifically, the descriptive text of the scale used to determine “user satisfaction” rating levels has been changed, as seen in the comparison of the two screens from the document, and the reference to people needing to perform additional research or see other results to get the information and remain satisfied has been removed.

In the new wording, therefore, these directions apply:

• Fails to Meet (FailsM) should be assigned to a result that completely fails to intercept the needs of all or most users. For example, the result might be off-topic for the query or might respond to an interpretation of the query that is virtually impossible.
• Slightly Meets (SM) should be assigned to a result that may be of little help for a prevailing, common, or reasonably secondary interpretation of the user’s intent, or to a result that may be helpful but for an interpretation of the user’s intent or the query that is rather rare or unlikely.
• Moderately Meets (MM) should be assigned to a result that is useful for any main, common, or reasonably secondary interpretation of the user’s intent or query.
• Highly Meets (HM) should be assigned to a result that is very useful for any main, common, or reasonably secondary interpretation of the user’s intent or query.
• Fully Meets (FullyM) is a special evaluation category, which applies only to queries with a clear intent to find a specific result and the corresponding specific result the user is looking for. These are, in the majority of cases, navigational queries or queries with a very specific request (finding an address, knowing certain information, and so on), which have no ambiguity of interpretation.

Google has also slightly modified the explanation of the criteria for the Quality Rater to consider in determining the rating, i.e., the extent to which the result:

• “fits” the query (fits the query, so to speak)
• is up-to-date
• Is accurate and reliable for the query
• satisfies the user

Previously, the criteria also included.

• is complete
• it comes from an authoritative source
• it satisfies the user or if the user would like to see other results.

What these guidelines are for (and why they affect us closely)

It is Tucker’s article again that summarizes why these guidelines are relevant and why, therefore, we need to keep track of the updates in place as well.

In fact, the guidelines are used by quality raters to evaluate the performance of Google’s various ranking systems, and “their evaluations do not directly affect ranking,” it is again reiterated.

More broadly, the guidelines share “important considerations about what content is useful to people when they use Google Search,” and it is no coincidence that the best practices on how to create useful, people-centric content summarize these concepts, offering them in a “quick” version to creators to help them self-assess their content for success in Google Search.

Although the quality raters’ evaluations do not directly affect rankings, they do provide feedback that helps Google improve its algorithms: it is therefore useful and important to spend some time looking at what the guidelines say and what new guidance Google is providing, to see if we can learn more about the ways that literally “drive” evaluations on sites and web pages and considerations about what needs to be done to intercept people’s needs and thus propose content that has a better chance of gaining organic visibility.