How to Make LLMs Shut Up

Greptile, an AI code review bot, faced challenges with generating too many low-quality comments on pull requests. The company implemented a clustering technique to filter out "nit" comments based on user feedback. By generating embeddings of past comments and comparing new comments to downvoted ones, Greptile successfully reduced noise in its output. This method increased the percentage of addressed comments from 19% to over 55% within two weeks of implementation, significantly improving the bot's usefulness for developers.