Ladders and Ranking systems

[Zetsumei]

Often websites that host a ladder, simply grab known ladder system of the web.

Be it rung based systems, elo, glicko, trueskill or any other variation. Its always important to understand how the system behaves with matches.

A good example can be found in elo, the most commonly used rating system which is used in for instance ESL ladder. In chess usually when playing ranked matches you only obtain a real ranking once you have played more than 20 matches. The simple reason for that is that in order to obtain high accuracy the change in ranking is relatively slow. By default the system got a very slow settle time in order to obtain a desired accuracy, which was more based on playing very regular matches with often more than one match per day.

Online based gaming is a fast changing game scenario this means, that within the period of 1 or 2 months a team can literary get half or double as good as they were before. Since most teams while playing daily practise matches, prefer to not play more than 1 ranked match a week. That means that within a small sample of say 4 matches you should already be able to adjust the scoring of the teams sufficiently.

In general ELO cannot keep up with these changes resulting in simply that playing the lowest rating team twice (very easy job) results in similar points to winning against say the highest rated team once.

This creates a serious issue of attacking low ranked teams for easy points, and especially new teams with a temporary base ranking.

A good ranking system states which team is stronger than which team, aka based on the ranking it should be able to predict a match' outcome with reasonable percentage of success.

If the system can not be used to predict the outcome of matches, then the actual given ranking often does not show an ordering of strength of teams. But rather an ordering of total game wins regardless of opponent, or other ways to basically play the system.

Be it by trying to not play any matches as much as possible in order to attain a certain position, or by playing a lot of weak teams to avoid playing strong teams.

-------------------------------------------------------------------------------

Whatever system is used teams will find a way to play matches in such a way that they get the highest ranking in the easiest method possible. This often at the cost of newer or weaker clans. So think out your ladder system well, for all I care use the ROLadder scoring systems since that has shown to work.

[Snuffeldjuret]

I thought the ROL2 system was really nice, although it didn't feel so much as an old ladder where you battle for points. To me it felt like a ranking. Nothing bad with that though, Way better than ESL ladder for instance.

[Base21]

I don't know how the elo works more precisely, but I think it's more of a system for ladders or rankings where there are ongoing encounters. It seems that if you have 20 matches before you even get startet to be ranked as a team/clan you will find it difficult to restart the ladder with this system as for each restart there must be 20 encounters for each team/clan to get a ranking result. It sounds like a waste of time if you want to do a season based ladder like for ex. Winter, Spring, Summer and Fall. or in even shorter periods.

Let's say 30 clans are in the ladder for a certain RO2 style like TE classic and 5 matches a day are played. It's just not going to work properly imo but I am sure someone with a little more experience can prove the opposite^^

How about a system that just takes win/draw/loss ratio and compares these to others. If you would like to challenge your opponents I think this is a bad idea but why would you anyway? To make it THE MOST FAIR and make the winner the most accurate winner there should be a mandatory playing schedule where you should not be able to cancel on your opponents. Very much like the German Bundesliga (soccer league).

*EDIT* In that way you will have a precise comparison on how each team did against each other, exact statistics and not some hocus-pocus.

[Zetsumei]

Originally Posted by Base21

I don't know how the elo works more precisely, but I think it's more of a system for ladders or rankings where there are ongoing encounters. It seems that if you have 20 matches before you even get startet to be ranked as a team/clan you will find it difficult to restart the ladder with this system as for each restart there must be 20 encounters for each team/clan to get a ranking result. It sounds like a waste of time if you want to do a season based ladder like for ex. Winter, Spring, Summer and Fall. or in even shorter periods.

Let's say 30 clans are in the ladder for a certain RO2 style like TE classic and 5 matches a day are played. It's just not going to work properly imo but I am sure someone with a little more experience can prove the opposite^^

How about a system that just takes win/draw/loss ratio and compares these to others. If you would like to challenge your opponents I think this is a bad idea but why would you anyway? To make it THE MOST FAIR and make the winner the most accurate winner there should be a mandatory playing schedule where you should not be able to cancel on your opponents. Very much like the German Bundesliga (soccer league).

*EDIT* In that way you will have a precise comparison on how each team did against each other, exact statistics and not some hocus-pocus.

Most game ladders actually let you get instantly get a score after playing a single match with an ELO system (when elo isn't converged it is essentially that the team with the best win/loss ratio is on top more on that later). While indeed those 20 matches perhaps should be made mandatory like with chess. 20 matches in clan terms is on average more than half a year meaning that most teams will probably have changed a lot in their skill. So indeed it easily shows an issue with ELO in its default configuration. Although logically Elo can be tweaked, and there are other systems.

Ladders are not supposed to be started and ended for a season they are supposed to run and continue to run indefinitely. In most sports, ladders are used to rather rank teams to say whether they can participate in a cup or tournament. Where the cup results get entered into the ladder for the next events.

Sure you can pick a winner based on a ladder on a seasonal basis but there is no reason to reset such a system unless there is a big reason why you assume the results filled in the ladder are incorrect.

You cannot base a ladder on pure win/loss/draw ratios. Simply because then the clan that challenges the most weak teams will end up as the winner (which is the main issue of an non converging elo system as well). The most active clan will show on top of the ranking and not the best clan. People can do this to the point that they can book a match for everyday just so they cannot be challenged by other clans.

Every ladder got rules that say when you can or cannot decline a challenge. But that doesnt fix issues with the default elo system.

Of course you can force clans to play a certain amount of matches in a fixed schedule but then its a league or cup instead of a ladder. Different tournament forms have different benefits. Where the benefit of a ladder is that it can showcase a global ranking with a very long and big group, allows teams to play matches with teams of similar strengths (instead of pounding newbs or playing against much stronger teams), and allows a team to somewhat decide for themselves how active they want to be.

Leagues in general are the most accurate form of determining what team is best but you need to work with fixed small team sizes and predetermined match dates. Tournaments have the issue that depending on the seeding one of the top teams could play early eliminating one of the other (hence why you often have seeding pools).

ELO works by essentially having 2 teams play each where both teams got a certain amount of points. If the score predicts a certain outcome say a 80% win and a team then wins with 90% one team will be raised a little and the other team lowered a little. In general this makes sense, however it can offset the position of a clan in relation to previous matches that were played. If you only move up a little and play a lot of matches this isn't an issue though.

Say Team A won vs B and C
Team D lost vs team E and F
And then team A loses vs team D.

With the last match in ELO team A loses points and team D gains points, although that makes sense this could mean that team A ends up lower than team B&C teams it just recently won against.

The ROLadder scoring system doesnt work by adjusting the teams that played a bit upwards or downwards it takes into account all previous results (with different weights) of the last 3 months. And places teams in a spot that in general fits that data best using a simple weighted least squares fitting method.

So in the previous example it would place teams in the order from best to worse in the following way: EF D A BC
Basically keeping the individual relations between matches.

ELO requires by default 20 matches to converge with small steps. the ROL system places you instantly in the position that fits the best with the played outcomes. When you play a lot of matches the ELO and ROL system show the same outcome. But in computer gaming when teams often play around 1 ranked match every 2 weeks, the instant convergence time of the ROL system vs ELO makes the ROL system much more accurate (especially as teams can grow in strength quite quickly).

When comparing a database of Ost matches with different rating systems the ROL system showed to be a clear winner.