Better approach than whitelisting

[rallfo]

I think the map and mutator whitelists are ideas with very good intentions, which I am assuming is predominantly to protect the legitimacy of perk progression and achievements.

However in practice they do not work. There will always be someone in the nether regions of the internet with a penchant for disassembly that will come up with some sort of crack/trainer that will bypass these checks or indeed just fake the network traffic signalling that an achievement has been awarded to the cloud.

What I am saying here is that people who really want to cheat will always find a way, whereas in the meantime us admins trying to run a legitimate server with our own mods and maps are saddled with the bureaucracy of getting everything whitelisted / the very principle that everything we do has to be "approved".

In fact I happen to know that there are things out there at the moment that will cause a cheater to instantly get awarded all achievements and perks in their steam account. What I think about those who use these things are unprintable, but this is the reality of the situation - the whitelisting does not stop cheaters and inconveniences everyone else.

My solution: (no I am not simply complaining!)

Given that the perk and achievement tracking is entirely centralized there is one way to make the system as cheatproof as it is ever going to be (i.e. it is not subject to the DRM paradox of stopping users interfering with software running on their own computer).

Simply resort to statistical methods at your end on the perk/achievement awarding progress variables of each player you already collect. Normal player progression will show a typical pattern that can be checked for, whereas cheaters will show a pattern far outlying this that could be flagged.

A concrete example: (bullpup part of commando levelling)

d(bullpup damage) / d(total playtime) will be within some standard deviation over that player's history* that can be ascertained by analysing legitimate player patterns.

If by some means a cheater maxes his recorded bullpup damage this value will spike outside of normal bounds and then the system could reset the progress and/or flag the account for VAC

What this system is effectively doing is checking that perk progression _is_ occurring at the same rate as a real player, and there is no way to "fake" this unless someone is willing to run a program that simulates this over the course of 120 hours, providing exactly the same data a real player would (and during which time their steam account would be unusable!).

Then the methodology of cheaters becomes irrelevant and whitelists become unnecessary, a double victory.

* In fact this value _would_ fluctuate, but predictably, reflecting varying levels of bullpup usage in different periods but I'm not typing out regression formulae in BBCode...

 

[gusone]

But that would mean the perk levelling maps you regulary run on your servers would not work.

 

[rallfo]

Thanks for that

Offtopic:
In case you are interested I obtained them _after_ the map whitelist, running them on a 6-man honey trap for my amusement that levellers waste hours not getting anywhere in apart from getting onto a list of steamids I don't want on my main server.

I fail to see how my sadistic admin private life is of any relevance to this discussion however, I assume you were just irked by the difference in numbers of our sigs

Insightful comments on the actual content of my post would be appreciated...

 

[Murphy]

For what it's worth, I'd rather play on his FF server than on a 30 man spam fest, but that's just my personal preference.

As for the idea, what if someone is simply on a roll? The main difference between levelling and just playing is that when levelling specimen can't get to you. If someone is good enough or has a good enough team he might get his perks locked and potentially a VAC warning (or worse) because he was doing well. If some deaths are required to keep the statistic "green" then levelling maps will just purge their players at certain (possibly random, if that's going to be a factor too) intervalls. E.g. by crushing them with a giant mover.

To make a long story short: I don't like white-listing either, but if people are annoyed by lame levellers and want them gone then white-listing is probably the only thing that can do that effectively.

 

[zYnthetic]

There would be a lot of math and even more probability involved. There's the basic math involved in calculating how long it would take to earn X amount of damage based off of ROF, but that's just the tip of the iceberg. There's many factors present that repeat in every game which close the window on the ability to inflict any damage, etc.. for variable lengths of time.

 

[rallfo]

On Murphy's point:
This proposal is to use a statistical method to check the growth of the Perk levelling progress variables already tracked and stored on the Steam cloud. These are basically what you see in the perk progress window, e.g. cumulative shotgun damage versus total playtime.

Tracking deaths is unnecessary, also the variables aren't updated frequently enough to do this sort of thing and therefore the issues you raise do not come into it.

We are basically checking to see if the moving average of the delta of each of these variables in inside a certain window (e.g. 3 standard deviations).

What the gist of it is, is if a player that was doing 10k damage shotgun per hour of playtime suddenly does a million it is likely that something is up, but these statistical methods are actually extremely good when presented with the data of the whole population.

Fundamentally if a player is stood in front of an invisible wall with the fire key macroed they will be doing significantly more DPS than even the most skilled player out there, and that is how this detection method will work, otherwise there would be no point to this level-cheating method in the first place.

What is being done is analogous to the fraud detection mechanisms present in financial systems. First we _assume_ that some fraud will be taking place - we cannot secure every computer involved and banks realized they cannot do this a long time ago - and instead we look for abnormal patterns in our data. In the same way we have to assume that cheaters will always be able to circumvent any software mechanisms in their client to prevent cheating because they already do. This approach will actually yield better real-world results than the current one of whitelisting (which is a mechanism attempting to secure the client and doomed to failure).

zYnthetic:
That there is a lot of math involved is never a reason not to do something!

Obviously this is not a complete solution, that can only be gleaned by analysis of real world levelling data, and what I posted was a naive but easily understandable approach because this is a forum predominantly populated by non-mathematicians. There are more sophisticated methods that would be more resiliant e.g. marginal distribution analysis over multivariate PDFs but that is actually an implementation detail.

IMO the fact that you can abstract this whole anti-cheating issue into an interesting math problem instead of some logistical problem involving verifying files would make this approach far more appealling to any programmer worth his function closures. You are taking the game up a notch and verifying the _behaviour_ of a distributed system instead of the system itself, since the latter is impossible.

 

[Murphy]

What the gist of it is, is if a player that was doing 10k damage shotgun per hour of playtime suddenly does a million it is likely that something is up

Such a drastic change isn't possible. Likely does not equal certain. False convictions with VAC bans, as you suggested must not take place or people wil lose faith in the system. Even things like Perk progress being lost as unjustified punishment might cause more trouble than it's worth. So this has to be extremely forgiving so a player who is doing too good for his own good can never be accused of cheating!

Fundamentally if a player is stood in front of an invisible wall with the fire key macroed they will be doing significantly more DPS than even the most skilled player out there, and that is how this detection method will work, otherwise there would be no point to this level-cheating method in the first place.

Really? Do I do more damage if I continuously blast into a horde of specimen in a scenario where they can't reach me or when I blast into a horde of specimen in a scenario where they can reach me but don't because I'm good?
In my opinion, not really. Also compare me playing beginner/clot-buster in BioticsLab and playing suicidal/bloated on KF-Farm. I'm going to do significantly more damage playing the latter! Is this cheating? Not really.
If you are going to allow mutators and (story-)maps (which is your goal, afterall) this might be even more drastic!

There is just not enough difference to completely rule out that a) levelling lamers get away with it and b) honest players get accused.

 

[Nanostrike]

But that would mean the perk levelling maps you regulary run on your servers would not work.

IMO, that pretty much sums up your creditability on this. Trolling or not, 'ol Gus has a very valid point there...

And I'm gonna bring up a point that Tripwire brought up when I used this sort of argument on the Steam forums: Where are all these programs that auto-max perks and achievements? Yes, I know they're out there. But they're kept so well-hidden that even Tripwire can't find them. Which means that the amount of people using them is insignificant at the moment.

Anyway, your system has a pretty big flaw: Folks could easily learn the maximum 'Okay' levels of damage to do in a certain period of time, and make sure to stay under that on their lame perk-leveling maps. If they botted or macro'd up something to fire JUST enough to not trip the system, they could easily start up a perk leveling map and just walk away while the computer levels them up overnight...

Long story short, whitelisting was the right thing to do. Any map that isn't an obvious perk-leveling map is getting added to the Whitelist the week or so after it's submitted. Even crappy maps. So no, it's not 'suffocating' the community or any of that nonsense. The only reason folks don't want whitelisting is BECAUSE they prefer maps that would NOT get the green light from Tripwire...

 

[gusone]

Of course if server admins didn't host the perk levelling maps in the first place we would not have this problem. Pity TWI cannot send an EMP ping/blast to servers breaking the EULA (by hosting these perk levelling abominations) that shuts them down and/or disables all perks for all maps on that server. I am more inclined just to shake my head and tut tut at the perk leveling kids (give 'em an inch they'll always take a mile, that's what kids do). Its the server admins that allow it to happen in the first place by hosting the maps who should be annihilated. I've no idea how that would work solo player - steal their PC in the dead of night maybe?

 

[rallfo]

IMO, that pretty much sums up your creditability on this. Trolling or not, 'ol Gus has a very valid point there...

Did you even read my response to that?
Do you not know what a honeytrap is?
And why do you think I'm interested in making the anti-cheating mechanisms better in the first place?!

I have never allowed anyone to cheat the levelling/achievement system on my server. My personal account has only 2 level 5 perks and under half the achievements!
In fact join my server: The perk system is entirely disabled!

Admin rivalry is one thing but claiming I am violating (a fictional) EULA is defamation. And it is frankly silly given I started this thread championing protecting it...

 

[rallfo]

On the relevant points raised:

Murphy you may be right but without the raw data that only Tripwire has available for analysis this is just guessing. Further even in that situation there WILL be a pattern that a computer can identify and use to weed out cheats even if it means using a neural network and profiling per player and per map.

What I am saying is that a TWI programmer should have a look into this line of thought.

Nanostrike: It is possible to get it so that "this pretty big flaw" IS the normal player behaviour. Again going back to the banking anti-fraud system the levels are set so that the amount of fraud that can be committed beneath the radar is not worthwhile.

The idea is that given this fake levelling program would have to run for the same amount of time anyway the player might as well level legitimately than not have access to his games for two days.

 

[Murphy]

even if it means using a neural network and profiling per player and per map.

Lol.

 

[Zetsumei]

Whitelisting is easy. Now while statistical match is for me atleast is one of the easier types of maths, it still takes time to create a system and processing power, but if a system depends on loads of variables, obtaining the correct statistics and correct dependability will get very difficult.

And most likely not too accurate, as how well you do depends on the players you play with and what settings you're playing at. Not to forget that players can get better overtime. Meaning you need atleast an average rating, a rating deviation (based on game settings), and a rating volatility. If you go into implementing all that together you end up with a system that is more enhanced that microsofts trueskill ranking system, which was created by several university math professors.

Even then most like the difference in scoring even taking everything in account will probably be within 3 times the standard deviation. And if someone actually does that regularly his own stats change as if hes a better player meaning his personal mean deviation and volatility will change and adapt to the cheating situation. Aka one would be free to actually continue to lame.

The only way then would be to take stats of all the players themselves, although generally the skills of players themselves are based upon the normal distribution. So if someone is in the top 10% it doesnt have to mean that all people there are exploiting.

Even though leveling maps are lame, they are purely lame because you cannot get hit. It doesnt change a thing in the damage you can do, the map doom is not a leveling map. Yet often during waves you just fire down the long hallways near a trader and kill them. Where it generally is if the specimen make it out of the hallway you're dead, if the team you play with is good then essentially that map gives the same effect as a leveling map. Although you actually need to work for it which makes all the difference, but for the stats optained its pretty much thesame.

 

[gusone]

In Defence of Doom it does allows a team to unleash hell down that long corridor which is rather good fun in moderation. One shouts the rallying call of "TIME FOR FIRE AND STEEL RAIN'" (check out the Badlands cover of 'Fire and Rain' for all you rockers) and the chaos then descends when you got 2-3 bullpups, firebug and others unloading. I just love the sound of near constant automatic multiple bullpups and the blood splattered wall at the end as a reward just makes that map for me.

On the relevant points raised:

As for stats... "Lies, damned lies, and statistics." Although it is interesting to hear what people have studied this week in school.

 

[zYnthetic]

That there is a lot of math involved is never a reason not to do something!

Zets actually explained it better. If it was just a lot of math it would be simple enough but there is far too much probability involved in events that could influence playtime if we were trying to verify the possibility a player has actually played long enough to be able to earn something. Implementing a security check based off probability carries the potential to deny a legit player from earning a reward. Simple solutions are usually the best ones.

 

[rallfo]

As for stats... "Lies, damned lies, and statistics." Although it is interesting to hear what people have studied this week in school.

Incidentally I do happen to hold an MSc in a mathematics honour and have moonlighted in a consultancy firm on the unsupervised transaction fraud detection system I kept making analogies to, so this idea was not plucked entirely out of the aether. By posting the most naive method I could contrive with the aim of keeping the discussion accessible to all I fell victim to people pointing out the obvious holes in it; the subject of my dissertation was in fact (topics relating to) automated fraud detection with multi-agent systems. No-one can spec a foolproof system of this nature in a forum post; this process would be a time-consuming iterative process.

Further I do believe such approaches can be wielded extremely effectively against this particular problem, have formally studied and worked on them, there are many multinationals that stake their very existence on the efficacy of such techniques, and therefore I know a system of this nature would ultimately be a far better and elegant solution than this whitelisting business if perfected.

I predict that we will start to see more and more common circumvention of the whitelisting system (since the verification is ultimately client side and therefore only security through obfuscation) and by choosing this path we're entering into a cat and mouse game with cheaters that cannot be won, the ultimate losers being everyone touched by the bureaucracy of the whitelisting process.

However you are most probably right about the pragmatic viability of developing such a system here and the resources and knowledge available to do it; but I would be more than willing to spec a reference implementation and there seem to be enough computer scientists kicking around here. My intention was to generate discussion on this issue and perhaps sow the seeds of an idea that may be revisited further down the line.

Addendum:
What are people's views on setting up a KF version of this?
It's what I am privately doing with the 6-man running KF-Levelling post whitelist
- that was the subject of gusone's ill-informed jibes

 

[Oldih]

Further even in that situation there WILL be a pattern that a computer can identify and use to weed out cheats even if it means using a neural network and profiling per player and per map.

As long as it plays inside the rules you laid for the pattern. It is true though that your average cheater is not that smart, but it doesn't take much to have many people falling under the flag of being suspcious by E.G. playing suicidal with small team and actually surviving it. Would mean quite high damageboost (presuming you have lvl4 perk(s)) within few hours and thus get under suspicous flag.

I could use Diablo II 'realm down' (aka temp ban) system as interesting example. It triggers if you do too many 'commands' in short period of time or rejoin\exit games with too hasty pace. It was mainly to counter botters (and of course, it was only very short term anti-function against them) but if you join a wrong game and hastily exit and hope to catch up with that earlier game, guess again. You got realmdown and cannot connect to bnet within the next one hour, or if you're having very, VERY bad luck, even for the next few days just because you fell on the generic pattern that used to exist that bots would do things that way. Sure it was effective method initially, but ceased to function so after the first few weeks and hurts only legit players in general (in case of that RD system).

 

[gusone]

Addendum:
What are people's views on setting up a KF version of this?
It's what I am privately doing with the 6-man running KF-Levelling post whitelist
- that was the subject of gusone's ill-informed jibes

http://forums.tripwireinteractive.com/showthread.php?t=33791

 

[rallfo]

I'd syndicate bans on this basis. Maybe it's time to set up a UTAN style system for KF... KTAN?