New Tribunal Features: Justice Review and Halls of Justice

Posted on at 2:58 PM by Moobeat
Some changes are on the way for the Tribunal, Riot's player driven system for punishing unsavory players. Two new features are being added: the Justice Review, where you can see what exactly your votes are doing to impact toxic players, and the Halls of Justice, a ladder of sorts where Tribunal users can compare their stats to others. Additionally, the small IP reward granted to summoners for judging cases is being removed.

Want to know more? Continue reading to get the full scoop and peruse some relevant red posts!

Here is the full announcement from Lyte about the new features and a preview of  what they are all about:
"Summoners!

It’s been almost a year and a half since we first launched the Tribunal and we’ve had some great results. Over 300,000 players (in NA and EU alone) who were previously punished have drastically improved their behavior as a result of Tribunal feedback and are now in good standing.

But there’s always room for improvement! Today, we’re making some changes to the Tribunal aimed at increasing its accuracy and providing players with more feedback about their voting record and effectiveness.

In our research, we’ve discovered that the vast majority of Tribunal voters don’t come to the Tribunal for IP – it’s a trivial amount for each case that players don’t really feel good about. Players come to the Tribunal day after day because they want to make a difference in the community. What’s even more important, though, is this: we discovered that the players who come to the Tribunal to make a difference are the most accurate voters and the players who come just for IP tend to be the least accurate.

In light of this research, we’ve decided to run a large-scale experiment to replace IP rewards with two brand new features in the Tribunal:
  • The Justice Review: We are adding a page where you can review individual cases you have judged and learn about your overall contributions. This includes information about how many toxic players you have helped remove, how many toxic games you have prevented, and your longest correct judgment streak. You can even view the case results for every single case you’ve voted on (even for cases you skipped).

  • The Halls of Justice: This is a new ladder for Tribunal users to see how their contributions compare to their peers. Players will get bonus points for getting streaks of cases correct and we’re excited to see and highlight the most positive contributors to the community!

If you have any questions, please feel free to post them in this thread and we’ll do our best to answer them!"


Here are a bundle red posts relating to the matter ( I'll be sure to add any new posts as they go up! ):

 Let's start out with the posts about when we this new Tribunal experience will launch.
RiotNinjaTabby on the when we can expect to see the new features implemented:
"We have finished up most of the work on these features and hope to get them to you very soon."
Similarly, Lyte chipped in by saying,
"We'll be launching these features in all regions in the near future. IP Rewards will still be handed out to players until the end of September, where we will turn it off and begin the full experiments."

Let's move on to a few posts about the reward aspect of the Tribunal, particularly why the IP reward was removed and if we can expect anything in its place.
 Lyte on why the IP reward was removed:
"To accurately run this experiment, we need to remove the IP rewards and measure whether Tribunal increases in accuracy. This experiment also helps figure out what percent of the population does Tribunal for IP versus doing them for these two new features.

We want to encourage players to use the Tribunal if they want to make a difference in the community and reduce the number of players who are trying to do the Tribunal just for IP."
Lyte going into even more detail on why the IP reward is being removed and reiterating that this is an experiment:
"If there's no motivation for you to use the Tribunal, then don't use it for now! The whole purpose of the experiment is to measure how changes in rewards changes Tribunal metrics such as:

1) The number of users that visit the Tribunal
2) The accuracy of the average Tribunal case
3) The efficiency of the Tribunal in removing toxic players

If you don't feel the current rewards are worth it for you, then don't participate. In Psychology, the new Justice Reviews reward players intrinsically. These rewards motivate players internally and tap their desires to make a difference in the community and become the pillars of our community.

The old rewards (IP) motivated players extrinsically. These rewards motivated players with tangible currency.

The experiment will help us determine how much of our Tribunal population is driven by extrinsic versus intrinsic rewards."
 RiotNinjaTabby on the possibility of other rewards for Summoners who participate in the Tribunal, such as the idea forum avatars:
"We definitely want to give users rewards for voting well and using their time to improve the community. We are tossing around lots of ideas for this, including feedback like yours. Forum avatars are worth consideration. We'd love to hear more ideas as well!"
Lyte responded to the same question with a very similar answer:
"There's potential for rewarding our most positive players, but this is something we're still discussing and figuring out."

Lastly, here are some responses to specific questions that fall don't fall into the two above categories:
Lyte responding to what if summoners don't want to be in a "ranked ladder" for judging cases and wish to remain anonymous. 
"We'll be keeping a close eye on all of these features to determine how they are affecting Tribunal usage, Tribunal accuracy, and Tribunal efficiency at removing toxic players from the game.

We're more than happy to tweak some features if it's better for the players. For example right now, only the Top 1000 Tribunal users will be shown in the ladder--everyone else has a personal ranking that is only visible on their Justice Review pages.

If we find that players do not want to be recognized on the ladder at all, this is something we will address. The main purpose of the experiment is to measure how changing rewards can change all of the metrics I mentioned above."
RiotNinjaTabby on if skipping casses ( not picking punish or pardon ) will mess up the correct cases streak:
"Nope, skip votes aren't part of the streak calculation"

RiotNinjaTabby confirmed the Halls of Justice & Justice Review stats will not be retroactive
"The tracking will start from when it's released."

73 comments

  1. Pretty cool features, but they could increase the IP to like 15 per case, what it's now is not that much and sometimes not worth the time :/

    ReplyDelete
    Replies
    1. Did you read? They are removing IP rewards.

      Delete
    2. This is exactly why they're removing IP rewards. So that people can't do it just for the rewards and spam punish.

      Delete
    3. If he is serious about people wanting to be apart of the tribunal to make a difference, then I owe this community an apology, because I always said LoL had the 2nd worse community in all of gaming, only second to CoD

      Delete
    4. It doesn't really make much sense for me. I mean when it is not the IP, they could at least give a special skin or something like this to the tribunal contributors. I think u need some form of a motivation to be there constantly, just a tiny little thing that changes the individual's gaming experience. A real judge wouldn't work at all, too, if u wouldn't give him his wages.

      On the other hand, the tribunal could really get more accurate now, when they start to remove all rewards. The changes are not bad, but I still haven't got a big motivation to participate in the tribunals.

      Delete
    5. They said they were contemplating and thinking of various rewards for people who do the Tribunal. Also, I think generally the reward is supposed to be seeing how much you have actually affected the community, and to see what your impact was. That's much more important to me than 5 IP per case, but other people are different, I suppose.

      Delete
    6. I've faced a lot less harassment on the LoL servers than while playing DotA, DotA 2, or HoN. Of course, I've faced a lot more harassment when my play sucks past 1 in the morning as well... >.>

      Delete
    7. I took all of the Tribunal cases I reviewed seriously but the real reason I was there was for the IP reward.

      I feel like Riot should include the IP reward again but change it so that you only get the IP if you're decision matched Riot's decision, not the majority of the community.
      The Tribunal is great, I love having a say if a player should not be punished or not.
      But it does take up a chuck of time, and now you don't even have any incentive for being there, because even if you judge a case and decide to punish someone Riot might think otherwise...
      The Tribunal isn't like jury duty, we don't HAVE to help Riot do their job...

      Delete
  2. wow...not that i did it much anyway but removing bonuses for spending time in tribunal... bad call

    ReplyDelete
  3. Hoping to see that on the BR Server... we need this as fast as humanly possible...

    ReplyDelete
  4. Anyone crying over the loss of IP for doing this is missing the point of why there doing this clearly.

    ReplyDelete
    Replies
    1. Yep, it's quite sad actually... :(
      Now at least we can separate the communty's golden boys from it's IP-hungry greedy hordes.

      Delete
  5. Terrible idea to remove IP from Tribunal. Everybody says they don't vote for the IP, but I bet most of them do it for the IP

    ReplyDelete
    Replies
    1. If you read the red post... They are doing the right thing on removing... This way you will only go there to do Justice.. Not for some extra IP... This way people will actually read what report says instead of just pressing punish...

      Delete
    2. I would want 20 or 30 cases to do, instead of 15.
      Cuz I thinks the tribunal is so "fun" !
      Why? Well, cuz I'm always being bullied in games..

      Delete
    3. ^ someone gets it

      Delete
  6. And when they voted for the IP they voted wrong. So how is that a terrible idea?

    ReplyDelete
    Replies
    1. Yep... just pressing punish will not always be the brightest idea...

      Delete
  7. Not gonna lie, I do it because I love the game and want to make the community better but by removing those IP rewards, trivial though they were, I probably won't be nearly as frequent. Time to go finish cases while we actually get something for caring.

    ReplyDelete
  8. There shouldn't be any reward for doing this other then removing people that ruin the community

    ReplyDelete
  9. Please, I slay trolls and that's enough of a reward for me. As long I don't get stuck with bad characters (not players) I am content. Vermin must be exterminated and that is every fair player's duty. So report, vote, sentence, execute!

    ReplyDelete
  10. This! This is what i want, i never cared for the IP at all.. Now all i want is for myself to see how acurate i can judge and to show off my good behaviour, and hopefully get others to be repectful and good :)

    ReplyDelete
  11. Well let's say it - by voting on the tribunal we are doing Riot's job. Not being rewarded for this is a shame... they'd rather give MORE bonus to people who give the winning vote (so the "punish" addicts wouldnt be much rewarded)

    @ward_baron

    ReplyDelete
    Replies
    1. Exactly, thats why I will not do the work for them anymore...

      Delete
    2. The intended reward for doing it should be the notification you receive later informing you of a successful verdict on a case. That way you won't get kids going there just to spam the punish button for free IP (people who critics of the Tribunal will insist make up the majority of the voters there).

      If you were going there just for the IP rewards which were miniscule then you were going for the wrong reason anyway. I certainly don't want people only interested in the Tribunal for their own selfish reasons to be responsible for clearing the community of scum.

      I'm very happy we'll be notified of what cases we were successful on myself. A little disappointed it's not retroactive so I can't see how well I was doing prior, but still very happy.

      Delete
    3. Good. Let the people that actually care about this game mold it into what it's surely becoming: #1 eSport in the world. Luckily, the ones that Judge are usually the ones that care the most about the game, so in the future it will be nothing but upstanding Judges who have cast out all of you riff raff that like to play a game to be rude to other human beings.

      Delete
  12. this

    doing -work- for ... -nothing-. wtf?

    ReplyDelete
  13. its doing work to get a better community but yeah... there should be sum reward for the hard work!

    ReplyDelete
  14. What if they decide to give better bonus from that justice rating?

    ReplyDelete
  15. i wanna be rewarded for spending 1 of my 24 hours a day reading chat logs from idiots and then it's for nothing (because tribunal is useless, you really think people get banned?)

    fyi im the original poster of this. bitch all you want, but its my opinion, i wont EVER go to tribunal again.

    ReplyDelete
  16. The post said they are thinking of rewards to give players, is everyone blind or what? Sure it's not IP, but it's still something unique you get for actually trying to make the community a better one. Plus there might be a skin or something for it, I'm sure.

    ReplyDelete
    Replies
    1. I would be more inclined to accept other rewards for what was already essentially free labor if IP wasn't next to impossible to make as is.

      Delete
    2. There definately needs to be some incentive.

      Delete
  17. Honestly, kind of sounds like I wouldn't really want you to be judging Tribunal cases, if all you are doing it for is the IP. And yes, people DO get banned by it. Why do you think there's so many 'OMG QQ I GOT BANNED!' threads on the forums?

    ReplyDelete
  18. I would wish that riot would make some long term rewards for those who have high justice rating, the old system is bad, but spending alot time to review cases should be rewarded also.

    ReplyDelete
    Replies
    1. Then people would just AFK between cases, like I do :P

      Delete
  19. Lmao you guys are doing this for the wrong purpose/goal its to get less trolls in the game so people can stop CRYING in forums after they get one in the game not for the ip thats just a bonus really if you do it for the ip and you are going to stop after these improvementes ok goodbye its not like there is not gona be enough people to check the tribunal LOL

    ReplyDelete
  20. dont much care if you use tribunal or not. i did it a few times and gave up after seeing the worthless amount of ip they give as a reward. i deal with pricks in almost everygame im in so i dont really see where the tribunal does anything. you want to see change? offer ip rewards for people that get votes for being awesome in game. reward leaders and guys that can laugh off a loss and have fun and you will see more of those people. even if theyre lying for the ip and screaming at their monitors irl.

    ReplyDelete
  21. ""i wanna be rewarded for spending 1 of my 24 hours a day reading chat logs from idiots and then it's for nothing""

    ????? Cant you only rate like 15 cases a day???? i know it may take me an hour then it says i reached my max per day. So your telling me it takes you 24 hours to read 15 cases,enough said.

    ReplyDelete
  22. I use the tribunal regularly and the IP points are barely worth spending the time on. At least this way we'll get people using the system to make the whole game better for people on a whole. The fact that 300'000 previous trolls have sharpened up their act as a result of the tribunal means that it's doing it's trick. And hell, if more people don't want to do it then that's fine. Go play against beginner bots if you're just looking for an easy way to get IP.

    ReplyDelete
  23. Love these changes. Thank you Riot!

    ReplyDelete
  24. I think if they want to keep IP rewards, they give them out based on high percentages, maybe even varying the bonus IP amongst the percentages, with the higher ones getting more.

    ReplyDelete
    Replies
    1. *same dude*
      Otherwise though, it's a great idea on Riot's part.

      Delete
  25. Love it.

    I would be interested to know if they plan on rewarding reviewer that do a good job over time- meaning you can't just rock up and spam punish to be rewarded.

    Either way though, I'll still be reviewing cases cause man, I hate jerks in this game.

    ReplyDelete
  26. The IP reward's so little it doesn't makes any different to me if they remove it. But I bet the number of ppl doing Tribunal will see a steep fall.

    if they read the post that they remove IP reward, that is.

    ReplyDelete
  27. YAY, now you can go for high ELO even in the tribunal, maybe I can manage to get high there at least =)


    Still not cool to remove this little and hardly noticable IP reward =(

    ReplyDelete
  28. This is all well and good, but the majority of smart "toxic players" are trolling pre, and post game. I think that sorting out this issue is more important than trying to weed out tribunal voters who are not voting accurately.

    ReplyDelete
  29. It is supposed to be community service, not a job.

    ReplyDelete
  30. To the third anon above me, he actually said it takes him 1 hour, not 24... "1 out of 24 a day".

    For the rest, it's just an experiment people. Let them try it, maybe they will see an improvement in votes, maybe they won't. If they see more accurate predictions it will be for the better, even without IP. Less trolls = more fun.

    ReplyDelete
  31. I stopped doing tribunal cause it seemed to make no difference. However with that bew feature I can track the changes I controbute to: absolutely what I wanted. Will definitly try it out in my free freetime (between gw2 abd lol + job training stuff it's a bit rare but I'll try x:).

    At those above... I wouldn't want you in tribunal nor in my games. Someone posts his opinion and you flame him for that? I don't agree with him as I couldn't care less about ip but seriously? I'm fine if you toss tribunal, please go ahead.

    ReplyDelete
  32. *Lyte experimenting again*
    "NOOOOOOOOOO"

    ReplyDelete
  33. I did tribunals for the IP, but I also tried to judge fairly as I didn't want to accidentally ban someone because I wanted to be greedy.

    ReplyDelete
  34. How about Doing Tribunals for lv30's already ive yet to judge a lv30 and thats where all the trolling is at...

    ReplyDelete
  35. The "Reward" is knowing and seeing the "toxic" aka instant-lose games you've just stopped.

    That's X more games where you would of instantly lost due to a troll. Frankly, I feel great just having them removed. Who gives a rats ass about IP any way, it's so minimal it's nothing. Better off just playing the game.

    ReplyDelete
  36. Well so long tribunal...

    ReplyDelete
  37. Well I won't be doing the Tribunal again. I did enjoy the little bit of IP that you got by judging cases correctly, but if I am not getting anything for judging cases I would rather play with these toxic players. I hope this experiment fails so we can be rewarded for this again.

    ReplyDelete
  38. agreed and on the note of not getting anything why would anyone say you get nothing with the ip reward being removed you still get to read the chat logs and amuse yourself with how people made asses of themselves

    ReplyDelete
  39. I never cared about ip gain from tribunal. I honestly did tribunal for so i personally would get less trolls, AM I REALLY THE ONLY ONE?

    ReplyDelete
  40. people are just thinking about the $. "What do I get for doing this?" is their first question, always. It's instinctive. They were tought since youth to be selfish and their interest comes before the community.

    It will be great once this experiment ends, Riot will see an increase in accuracy in votes since all the cheap greedy players were scattered and lost interest in working for "free" and the ones that did it before with a righteous purpose will stay and improve the game, even for those cheap, dirty and greedy ones.

    If you wanna do it in a selfish way, do it so the number of trolls in your team will decrease in the future.

    And they already said there might be some exclusive stuff like forum avatars.

    ReplyDelete
    Replies
    1. i will probaly spam punish now to rid the community of the trolls.
      anyway since when you get in the tribunal you server punish

      Delete
    2. deserve to be punished^^

      Delete
  41. for me, as i do my job thoroughly, it takes a good amount of time for me to judge cases. But i was doing it principally because of the reward, i was taking time on my playing time just for that.
    Judging is fun ? After a lot more than 100 cases judged it's not amusing anymore as you find pretty much always the same type of persons. Same type of persons, which feel like judging cases don't seems to dimished really the number of bad persons on lol.

    In most game as i play in a guild i don't really suffer that much from bad players so without the reward i don't feel the need to judge people that much.

    ReplyDelete
  42. i would have to say getting rid of the ip reward was a bad call by riot. i don't go to work just to see a smile on your face. I can give two shits about you. it about the money, something riot should understand. Now what they should have done is make another reward for judging the players. Also whos ever head of their marketing department needs to go back to school.

    ReplyDelete
  43. and other thing if the ip was such a bad thing because players were giving out banneds because it easyier to get ip riot could have given out ip just for help not based on judgemnet

    ReplyDelete
  44. Very good info. Lucky me I discovered your blog by chance (stumbleupon).
    I have saved as a favorite for later!

    Also visit my web site ... cheap ralph lauren polo

    ReplyDelete
  45. I loved as much as you will receive carried out right here.
    The sketch is attractive, your authored material stylish.
    nonetheless, you command get got an impatience
    over that you wish be delivering the following. unwell unquestionably come
    more formerly again since exactly the same nearly a lot often inside case you shield
    this hike.

    Here is my web blog - supra thunder

    ReplyDelete