Tournesol's model

From Tournesol
Jump to navigation Jump to search

Tournesol collects pairwise content comparisons along different quality criteria, and infers individual and global scores based on such comparisons. To do this, Tournesol's model combines the Bradley-Terry model and the Licchavi framework for robust personalized collaborative learning.

Note: A paper on Licchavi is currently being finalized. It will be available within a month.

Interpretation of point differences

Upcoming...

Basic mathematical formulation

Denote the set of contributors and the set of videos on Tournesol. For simplicity, for now, we assume that there is only one quality criterion.

Each contributor then provides a dataset of ratings, each of which is of the form , where are two videos to be compared, and is the rating provided by the contributor. The value means that the contributor vastly prefers to .

By slightly generalizing the Bradley-Terry model, we assume that that each contributor implicitly assigns a score to video . Intuitively, we then assume that the odds that contributor rates above is exponentially large in the difference between the implicit scores of the two videos. More formally, we assume that the law of rating given the implicit scores of contributor is given by the probability density function .

Assuming that the contributor's ratings are independent (conditionally to the videos selected and and to the parameters ), the negative log-likelihood of the dataset is then given by .

As proposed by the Licchavi framework, we introduce global scores for all videos . We then penalize the discrepancies between the global scores and the individual scores, as well as a regularization on the global scores to guarantee the uniqueness (and robustness) of global scores. This leads us to define the following global loss: .

The weights are defined by , where is the number of ratings of video by contributor . They initially increase linearly in , as contributor provides more ratings, but then saturate at 1, thereby giving the contributor a bounded maximal voting power.

Note that the global loss is convex. We currently solve it using gradient descent, which currently takes us less than 10 minutes on a CPU, for ~ 5,000 ratings. We are also investigating solutions to scale the optimization of the loss function to solve it for millions or billions of ratings.

Currently the hyperparameters are set as , and .

Resilience to a small number of malicious contributors

A single contributor with 8 ratings can only make the scores go up to 1.364.
A single contributor can have an effect of at most on the scores. For , and ratings, this equals ~0.364. This is why some scores of this video are stuck at 1.364.

This implies that, currently, a voter can affect the global score on a video by at most points. In fact, a contributor that provides ratings would only be able to influence global scores by 1/4 point.

Larger values of increase the robustness of our model to malicious contributors. As the number of contributors grow, we plan to increase this ratio as well.

Research

Tournesol's model is currently being studied both theoretically and empirically. To contribute to this research, please reach out to Lê Nguyên Hoang, e.g. on Discord.