In Build Recommender System with LLM-based Reranker, we explored turning a user’s past interests into a query and using a reranker to rank candidate items. With the emergence of decision models such as Jev, recommender systems have another option: treat “Will this user like this item?” as a judgment task and rank items directly by the probabilities returned by the decision model.

In 2025, we benchmarked the performance of text embedding models in similarity-based recommendations. Within six months of that benchmark's publication, Alibaba Cloud and Google launched a new generation of open-source text embedding models: qwen3-embedding by Alibaba Cloud and embeddinggemma by Google. Recently, the gorse-bench tool also added benchmark features for text embedding models. In this post, we use gorse-bench and the playground dataset to conduct a comprehensive evaluation of popular open-source text embedding models.