Server KF2 (or any Unreal Engine 3) server on RedHat/CentOS/Rocky/Alma Linux DDoS defense with the help of firewalld

[o2xVc3UuXp0NyBihrUnu]

iptables -A INPUT -p udp -m udp --dport 7777:7779 -m connlimit --connlimit-above 3 --connlimit-mask 20 --connlimit-saddr -j DROP

This does not work for me.

I'm not sure why the -m (match) option is here, check with baz. I don't use it and it works. Check that your servers are indeed listening on ports that are being defended here (netstat -plunt). Check your iptables is installed and functional. Check if the rules were indeed added by checking the appropriate chain. There was also a command shared a few pages back that used the "watch" command to see how the packets go through the rules in real time, you should see the numbers there if it works.

 

[soundguy]

So may i know what is your exact iptables query? Im noob to this and even if there is some documentation avaiblable, it does not say what i need to use or not. I am trying to achieve this on OpenWRT router, but it could be a problem, that router uses for port forwarding also iptables, but these phrases and lines are hidden, co they can collide - my opinion.

Worst problem is i cant achieve running server on different port than 7777. Is it possible to change only this port without changing others from defaults 8080, 20560, 27015 and keeping them as they are? I have tried changing port in config.. but server does not appear in steam server list then, people cant connect using friends list or using open ipaddress/port phrase.. Im really consterned. Mostly attacks is directed to 7777 and im unable to change this port. Tried in config, then in config and bat file used to run server.. and of course port forwarding that port. Or even tried to port forward 8888 from outside to 7777.. etc. with keeping default 7777 port, then changing it to 8888, using INI file or command line separately to notice about port number, or both altogether. I can join from lan.. not seeing server in LAN servers, but using OPEN localipaddress/port. But players from outside cant. Also tried to specify multihome. And double checked what is the connect phrase in server window, that was correct.. It even does not appear in server list, checked also proper connect to hydraapi.

I know its not OPEN ipaddress/port but when i use the correct char : , forum changes it to smile face.

 

[baztheallmighty]

So may i know what is your exact iptables query? Im noob to this and even if there is some documentation avaiblable, it does not say what i need to use or not. I am trying to achieve this on OpenWRT router, but it could be a problem, that router uses for port forwarding also iptables, but these phrases and lines are hidden, co they can collide - my opinion.

Worst problem is i cant achieve running server on different port than 7777. Is it possible to change only this port without changing others from defaults 8080, 20560, 27015 and keeping them as they are? I have tried changing port in config.. but server does not appear in steam server list then, people cant connect using friends list or using open ipaddress/port phrase.. Im really consterned. Mostly attacks is directed to 7777 and im unable to change this port. Tried in config, then in config and bat file used to run server.. and of course port forwarding that port. Or even tried to port forward 8888 from outside to 7777.. etc. with keeping default 7777 port, then changing it to 8888, using INI file or command line separately to notice about port number, or both altogether. I can join from lan.. not seeing server in LAN servers, but using OPEN localipaddress/port. But players from outside cant. Also tried to specify multihome. And double checked what is the connect phrase in server window, that was correct.. It even does not appear in server list, checked also proper connect to hydraapi.

I know its not OPEN ipaddress/port but when i use the correct char : , forum changes it to smile face.

Your explanation of your setup is confusing. I assume you are using OpenWRT as a router and you have another physical device as a server. I also assume you are port forwarding to your server.

Just a FYI I have never used OpenWRT. Based in the 5 minute read of the documentation you will need to add the rules using nft assuming you are using a newer version. That information came from here: https://openwrt.org/docs/guide-user/firewall/netfilter_iptables/netfilter_management

OpenWRT documentation appears to be here: https://openwrt.org/docs/guide-user/firewall/netfilter_iptables/netfilter_examples

If OpenWRT is an older version and is using iptables you will be able to use these two commands to view all of the rules on your router:
iptables -xnv --line-number -L
iptables -xnv --line-number -L -t nat

If OpenWRT is using nft you will be able to use these this command to view all of the rules on your router:
nft list ruleset

It looks like the rules will need to be translated into nft. You would normally put the rules into the FORWARD table. This is different for nft, you will have to see what is setup on the device. You should be able to the iptables-translate command to translate the rules assuming OpenWRT has the command iptables-translate installed.


List of very generic documentation around nft:

Firewall documentation

Firewall documentation

openwrt.org

Chapter 10. Getting started with nftables Red Hat Enterprise Linux 8 | Red Hat Customer Portal

Access Red Hat’s knowledge, guidance, and support through your subscription.

access.redhat.com

How to Use nftables

In this guide you will learn about what nftables is and how it differs from iptables, plus you""ll get a look at how to use and create tables, rules, and chains.

www.linode.com

To change the port of the server I do that mostly in the startup line. e.g KFServer.exe KF-Cube?Game=KFGameContent.KFGameInfo_Endless?Difficulty=3?ConfigSubDir=CUBE2 -QueryPort=xxxx -Port=yyyyy -WebAdminPort=zzzz

Documentation (see Command Line Launch Options): https://wiki.killingfloor2.com/index.php?title=Dedicated_Server_(Killing_Floor_2)

 

[o2xVc3UuXp0NyBihrUnu]

So may i know what is your exact iptables query?

I can't, I don't use iptables, I rely on firewalld, which is managed with firewall-cmd command-line tool and what I use is shared in the original post at the beginning of it.

Im noob to this and even if there is some documentation avaiblable, it does not say what i need to use or not. I am trying to achieve this on OpenWRT router, but it could be a problem, that router uses for port forwarding also iptables, but these phrases and lines are hidden, co they can collide - my opinion.

They absolutely can, e.g. first run iptables -L -v to get an idea of the rules you already have in each chain. If it's a router you are looking to modify the FORWARD chain, as baz does, although INPUT may also work. In the FORWARD chain look where it would make sense to insert/add your rule, depending on what you already have.

Worst problem is i cant achieve running server on different port than 7777. Is it possible to change only this port without changing others from defaults 8080, 20560, 27015 and keeping them as they are?

The webadmin console (BTW -- don't expose it to the internet, it's vulnerable) is independent, but other ports are derived automatically from the game port. The game port is specified either in the starting command line or in the config file, if you run it on Windows, it's PCServer-KFEngine.ini, parameter Port=7777. Change it to something else in any text editor (e.g. notepad) and restart the server. But you will also need to reflect this configuraiton on your router for port forwarding to work, unless you forward everything (which you shouldn't for security reasons).

I have tried changing port in config.. but server does not appear in steam server list then, people cant connect using friends list or using open ipaddress/port phrase.. Im really consterned.

This is a typical port forwarding problem. Understand this -- if you have any router in front of your KF2 server gatekeeping it from the Internet, this router will need to forward the exact ports on it (e.g. 7777) to appropriate ports on the KF2 server. Ensure that you forward all appropriate game ports, which you can learn from the output of the 'netstat -nao' command on Windows.

 

[o2xVc3UuXp0NyBihrUnu]

I'd like to report that on one of the VMs I own I changed the game ports to the 27,815+ range back in February 2023. The game query port gets assigned automatically by adding 19,238 to it so they are in the 47,053+ range.

Previously I've experimented with changing the default ports, but only in the vicinity of the standard for UE3 UDP 7777. They were always rediscovered by the criminals just a few days later after the change and DDoS resumed.

This time I run them for 2+ months and still no attacks.

If your servers are attacked, please try to change the ports to weird ranges, start with, say, 30,000+ and report here if the DDoS resumes. I would be very much interested to learn either way, if it resumes or not, I have a couple of theories I need to verify.

While the solution in the OP works 100% fine for me without any issues whatsoever, even the server memory consumption doesn't noticeably grow and I can run them perfectly fine for weeks without any crashes or memory bloat whatsoever, as well as no reports on false positive IP bans from the regular players, I'm curious to learn why KF2 servers hosted on this particular VM with weird ports are not under attack for so long.

 

[GenZmeY]

I'd like to report that on one of the VMs I own I changed the game ports to the 27,815+ range back in February 2023. The game query port gets assigned automatically by adding 19,238 to it so they are in the 47,053+ range.

Previously I've experimented with changing the default ports, but only in the vicinity of the standard for UE3 UDP 7777. They were always rediscovered by the criminals just a few days later after the change and DDoS resumed.

This time I run them for 2+ months and still no attacks.

If your servers are attacked, please try to change the ports to weird ranges, start with, say, 30,000+ and report here if the DDoS resumes. I would be very much interested to learn either way, if it resumes or not, I have a couple of theories I need to verify.

While the solution in the OP works 100% fine for me without any issues whatsoever, even the server memory consumption doesn't noticeably grow and I can run them perfectly fine for weeks without any crashes or memory bloat whatsoever, as well as no reports on false positive IP bans from the regular players, I'm curious to learn why KF2 servers hosted on this particular VM with weird ports are not under attack for so long.

An interesting observation
This range usually contains ephemeral ports that are used temporarily by the client side
This only exists at the convention level, there are no technical restrictions, so they can also be used even though no one usually does this
It's hard to say why the attacker does not attack this range, but I think he can modify his tool for such a case

 

[jwshields]

Just to chime in here.

iptables -A INPUT -p UDP --dport 7700:8700 -m ttl --ttl-gt 240 --jump DROP

replace the dport range with whatever ports your servers are listening on for game connections.
I have noticed quite a dramatic drop in malicious activity since implementing this rule. Even on the steam ports. Looking over network traffic logs and graphs, inbound bytes per second versus outbound has leveled off and I am not seeing signs of reflection attacks (In my charts, it's signified by outbound not being 10x higher than inbound, in terms of bytes per second).

This all is not to say it's fixed or that the attacks won't adapt. But this mitigation has helped my servers tremendously.
I've discussed this with other various server ops in great detail over Discord lately, but anyway. Just wanted to throw my hat in the ring.

And fwiw, I run my servers on default ports, or adjacent ports.

 

[baztheallmighty]

Just to chime in here.

iptables -A INPUT -p UDP --dport 7700:8700 -m ttl --ttl-gt 240 --jump DROP

replace the dport range with whatever ports your servers are listening on for game connections.
I have noticed quite a dramatic drop in malicious activity since implementing this rule. Even on the steam ports. Looking over network traffic logs and graphs, inbound bytes per second versus outbound has leveled off and I am not seeing signs of reflection attacks (In my charts, it's signified by outbound not being 10x higher than inbound, in terms of bytes per second).

This all is not to say it's fixed or that the attacks won't adapt. But this mitigation has helped my servers tremendously.
I've discussed this with other various server ops in great detail over Discord lately, but anyway. Just wanted to throw my hat in the ring.

And fwiw, I run my servers on default ports, or adjacent ports.

An interesting approach. Easily bypassed with a little bit of research. In theory you could drop this down to 129 as the default windows ttl is 128 and linux is 64.

 

[jwshields]

An interesting approach. Easily bypassed with a little bit of research. In theory you could drop this down to 129 as the default windows ttl is 128 and linux is 64.

Yeah - I was examining a number of packet captures I did. Most of the steam api reflection packets had TTLs around 110-140. But all of the malicious packets that I saw to the game server ports (7777, etc) - all had abnormally large TTLs. I personally try to err on the side of caution, but I feel pretty confident in saying that my iptables example could be lowered to 129 like you suggested. That said, I have seen legitimate, non-kf2, packets in the wild with some TTLs above that. So, for me, could it block legitimate players? Quite possibly. What's the likelihood of that happening? I'd bet the probability is in the range of... one to "I'm a prince in kenya and you've been named as an heir to a generous amount of money" being true.

And yes, of course, easily bypassed. But if these attacks are by skids with mommy's credit card renting a stresser or using loic, I'd wager they're not gonna be that bright.
The reason I say that is because almost all of the attacks I captured from my servers were incredibly generic, unchanging - which makes me think a lot of it is scripted or automated; sorta like shotgunning for WP exploits, as skids do.
And, I guess to me, it seemed to be an attempt to DDoS by way of resource exhaustion more than anything else. But that's just IMO.

 

[o2xVc3UuXp0NyBihrUnu]

This DDoS may have ceased worldwide at this point. None of my servers have been getting hit for more than 1 week. Maybe TWI fixed it or did something and didn't say anything.

 

[jwshields]

I just noticed similar on my VPS. Connection rates seemed to die down a few days ago, (but my CPU start spiking for a reason I've not tracked down yet) - nothing sus in the logs.
Updating the game and restarting my servers has seemingly brought CPU back down.
Will keep monitoring it though.

 

[baztheallmighty]

This DDoS may have ceased worldwide at this point. None of my servers have been getting hit for more than 1 week. Maybe TWI fixed it or did something and didn't say anything.

Not fixed. Just lucky that no one is doing amplification attacks at the moment..

 

[jwshields]

View attachment 2337472

Not fixed. Just lucky that no one is doing amplification attacks at the moment..

The recent game update included a newer steamclient.dll/steamclient.so file which fixed the reflection/amplification attack bug.

 

[baztheallmighty]

The recent game update included a newer steamclient.dll/steamclient.so file which fixed the reflection/amplification attack bug.

Not sure where you got that information from but it's wrong.
A quick test of sending 11 packets to my server resulting in 10846 packets being returned from my server.

 

[o2xVc3UuXp0NyBihrUnu]

View attachment 2337485
Not sure where you got that information from but it's wrong.
A quick test of sending 11 packets to my server resulting in 10846 packets being returned from my server.

Yeah, I just reproduced an experiment I did back in 2021 with the help of netcat:

ncat -u 5.161.149.88 7779

It still responds with tons of nonsense in reply to any input and roughly at the same rate as I remember from 2021. So the amplification vulnerability is definitely still there. Still, something has happened, because I don't see the attacks ongoing. I've been under these attacks non-stop since 2021 with varying intensity, but not anymore. Why did they stop?

 

[jwshields]

Ah, my apologies, I had read in Discord that the new release had an updated steamclient which was said to fix the amplification attacks, I took that at face value, but never validated it, because the attacks entirely flatlined after I started dropping high TTL packets.

 

[subconscious]

Our servers are getting hit by the amplification attacks again, after months of not needing to filter our network, so be warned that whoever is doing this amplification attack is back at it, now its not hard to filter these attacks but just the fact that this stuff is still happening baffles my mind, please let me know if any of u are getting these “attacks” aswell recently again.

 

[o2xVc3UuXp0NyBihrUnu]

Yes. Currently 184 /20 IP ranges are on the ban list, which is about a typical load from a few months back.

PS Also, I expect to depart this forum in the next month or so. It's going to be up to people like you, subconscious, baztheallmighty, etc to help KF2 community server hosters with this BS.

 

[subconscious]

The TTL solution seems to be taking out the attack completely, let’s hope this ddosing bastard is not adapting on this, since ipbanning is endless, my iptables reached up to 7k banned ips in a lets say 2 week timespan..

 

[Ruuvi]

Some observation, DDOS started after publishing the server in gametracker..
I maked list of ip from which DDOS is produced.
This ip list is constantly auto updated.

-->kf2.tix.su/firewall.log