New Tribunal Features: Justice Review and Halls of Justice

Some changes are on the way for the Tribunal, Riot's player driven system for punishing unsavory players. Two new features are being added: the Justice Review, where you can see what exactly your votes are doing to impact toxic players, and the Halls of Justice, a ladder of sorts where Tribunal users can compare their stats to others. Additionally, the small IP reward granted to summoners for judging cases is being removed.

Want to know more? Continue reading to get the full scoop and peruse some relevant red posts!

Here is the full announcement from Lyte about the new features and a preview of  what they are all about:

"Summoners!

It’s been almost a year and a half since we first launched the Tribunal and we’ve had some great results. Over 300,000 players (in NA and EU alone) who were previously punished have drastically improved their behavior as a result of Tribunal feedback and are now in good standing.

But there’s always room for improvement! Today, we’re making some changes to the Tribunal aimed at increasing its accuracy and providing players with more feedback about their voting record and effectiveness.

In our research, we’ve discovered that the vast majority of Tribunal voters don’t come to the Tribunal for IP – it’s a trivial amount for each case that players don’t really feel good about. Players come to the Tribunal day after day because they want to make a difference in the community. What’s even more important, though, is this: we discovered that the players who come to the Tribunal to make a difference are the most accurate voters and the players who come just for IP tend to be the least accurate.

In light of this research, we’ve decided to run a large-scale experiment to replace IP rewards with two brand new features in the Tribunal:

• The Justice Review: We are adding a page where you can review individual cases you have judged and learn about your overall contributions. This includes information about how many toxic players you have helped remove, how many toxic games you have prevented, and your longest correct judgment streak. You can even view the case results for every single case you’ve voted on (even for cases you skipped).

• The Halls of Justice: This is a new ladder for Tribunal users to see how their contributions compare to their peers. Players will get bonus points for getting streaks of cases correct and we’re excited to see and highlight the most positive contributors to the community!

If you have any questions, please feel free to post them in this thread and we’ll do our best to answer them!"

Here are a bundle red posts relating to the matter ( I'll be sure to add any new posts as they go up! ):

 Let's start out with the posts about when we this new Tribunal experience will launch.

RiotNinjaTabby on the when we can expect to see the new features implemented:

"We have finished up most of the work on these features and hope to get them to you very soon."

Similarly, Lyte chipped in by saying,

"We'll be launching these features in all regions in the near future. IP Rewards will still be handed out to players until the end of September, where we will turn it off and begin the full experiments."

Let's move on to a few posts about the reward aspect of the Tribunal, particularly why the IP reward was removed and if we can expect anything in its place.

 Lyte on why the IP reward was removed:

"To accurately run this experiment, we need to remove the IP rewards and measure whether Tribunal increases in accuracy. This experiment also helps figure out what percent of the population does Tribunal for IP versus doing them for these two new features.

We want to encourage players to use the Tribunal if they want to make a difference in the community and reduce the number of players who are trying to do the Tribunal just for IP."

Lyte going into even more detail on why the IP reward is being removed and reiterating that this is an experiment:

"If there's no motivation for you to use the Tribunal, then don't use it for now! The whole purpose of the experiment is to measure how changes in rewards changes Tribunal metrics such as:

1) The number of users that visit the Tribunal
2) The accuracy of the average Tribunal case
3) The efficiency of the Tribunal in removing toxic players

If you don't feel the current rewards are worth it for you, then don't participate. In Psychology, the new Justice Reviews reward players intrinsically. These rewards motivate players internally and tap their desires to make a difference in the community and become the pillars of our community.

The old rewards (IP) motivated players extrinsically. These rewards motivated players with tangible currency.

The experiment will help us determine how much of our Tribunal population is driven by extrinsic versus intrinsic rewards."

 RiotNinjaTabby on the possibility of other rewards for Summoners who participate in the Tribunal, such as the idea forum avatars:

"We definitely want to give users rewards for voting well and using their time to improve the community. We are tossing around lots of ideas for this, including feedback like yours. Forum avatars are worth consideration. We'd love to hear more ideas as well!"

Lyte responded to the same question with a very similar answer:

"There's potential for rewarding our most positive players, but this is something we're still discussing and figuring out."

Lastly, here are some responses to specific questions that fall don't fall into the two above categories:

Lyte responding to what if summoners don't want to be in a "ranked ladder" for judging cases and wish to remain anonymous. 

"We'll be keeping a close eye on all of these features to determine how they are affecting Tribunal usage, Tribunal accuracy, and Tribunal efficiency at removing toxic players from the game.

We're more than happy to tweak some features if it's better for the players. For example right now, only the Top 1000 Tribunal users will be shown in the ladder--everyone else has a personal ranking that is only visible on their Justice Review pages.

If we find that players do not want to be recognized on the ladder at all, this is something we will address. The main purpose of the experiment is to measure how changing rewards can change all of the metrics I mentioned above."

RiotNinjaTabby on if skipping casses ( not picking punish or pardon ) will mess up the correct cases streak:

"Nope, skip votes aren't part of the streak calculation"

RiotNinjaTabby confirmed the Halls of Justice & Justice Review stats will not be retroactive

"The tracking will start from when it's released."