Workshop appears to be broken [dedicated server]

[baztheallmighty]

Hi all,
So it appears that workshop downloads are broken on windows. I have tested this on two completely new installs. No love. Anyone have any solutions?

My test map has been but all other mods and maps are not working also.

Steam Workshop::Zed's Diner

steamcommunity.com

Examples of the errors i am getting:
[0021.85] Warning: Warning, Workshop: Unable to download workshop item 1102122008
[0021.85] NetComeGo: Checking item 1126487682 for updates.
[0021.85] NetComeGo: Item state is 0.

and

[2023-02-20 21:54:03] [AppID 232090] Workshop depot not defined, using AppID itself
[2023-02-20 21:54:03] [AppID 232090] Loaded workshop items in "C:\KF2Server\Binaries\Win64\steamapps\workshop" (0 installed, 0 needed)
[2023-02-20 21:54:08] [AppID 232090] Download item 1102122008 requested by app
[2023-02-20 21:54:08] [AppID 232090] Starting Workshop download job (downloading )
[2023-02-20 21:54:09] [AppID 232090] Detected workshop change : changed cached item 1102122008, new manifest 9113756988096896329
[2023-02-20 21:54:09] [AppID 232090] Workshop update job started while app is running, commit : yes
[2023-02-20 21:54:17] [AppID 232090] Update canceled: Failed to download manifest 9113756988096896329 for depotID 232090 (Not Logged On) (Manifest unavailable) "cache5-par1.steamcontent.com/depot/232090/manifest/9113756988096896329/5"
[2023-02-20 21:54:17] [AppID 232090] Download item 1102122008 result : Failure
[2023-02-20 21:54:17] [AppID 232090] Finished Workshop download job : Manifest unavailable
[2023-02-20 21:54:37] [AppID 232090] Updating workshop item details failed with Timeout


edit:
This is for dedicated server workshop downloads.

 

[o2xVc3UuXp0NyBihrUnu]

There's a solution but only for Linux. It involves replacing the steamclient.so file in /kf2server/Binaries/Win64/lib64/ path with an older version of the steamclient.so. I was given this file by someone, no idea where did they get the older version of steamclient files from.

I know you host on windows from previous conversations, and your paths hint at that, so I would try playing with a similar file in a similar folder and try replacing it with this same file of other older steam client versions.

 

[baztheallmighty]

So i figured it out with a bit of luck. Turns out the new version updates your config files every time you launch the game. Specifically it replaces the [IniVersion] line in the config files:
pcserver-kfinput.ini
PCServer-KFEngine.ini
PCServer-KFGame.ini
KFWebAdmin.ini
KFWeb.ini
PCServer-KFSystemSettings.ini
KFAI.ini

This is an example of PCServer-KFGame.ini it put into my config. THIS DOESN'T WORK!!
[IniVersion]
0=1571393434.000000
1=1670434420.000000
2=1571393434.000000
3=1571393566.000000

I am unsure which config file actually breaks the workshop as i gave up troubleshooting. I just wrote a script to do a find and replace across all of my config files with known working values. Every time the game starts the bad values are put back into the config files. So the script has to be run before every launch of the server.



The script to fix the issue.

$KFInput=@"
[IniVersion]
0=1571397034.000000
1=1622559055.000000
2=1571397034.000000
3=1571397139.000000
"@

$KFEngine=@"
[IniVersion]
0=1571393434.000000
1=1670434418.000000
2=1571393434.000000
3=1571393539.000000
Patch=1059
"@

$KFGame=@"
[IniVersion]
0=1571397034.000000
1=1655400079.000000
2=1571397034.000000
3=1571397166.000000
"@


$KFWebAdmin=@"
[IniVersion]
0=1571397164.000000
"@

$KFWeb=@"
[IniVersion]
0=1635761032.000000
"@

$KFSystemSettings=@"
[IniVersion]
0=1571397034.000000
1=1571397155.000000
2=1571397034.000000
3=1571397164.000000
"@

$KFAI=@"
[IniVersion]
0=1571397167.000000
1=1571397155.000000
"@

cd C:\KF2Server\KFGame\Config
Get-ChildItem  -Recurse pcserver-kfinput.ini          | % { (get-content $_.fullname -raw ) -replace "(?ms)\[IniVersion\](.*)\r\n\s*\r\n",$KFInput | Set-Content -Path $_.fullname}
Get-ChildItem  -Recurse PCServer-KFEngine.ini         | % { (get-content $_.fullname -raw ) -replace "(?ms)\[IniVersion\](.*)\r\n\s*\r\n",$KFEngine | Set-Content -Path $_.fullname}
Get-ChildItem  -Recurse PCServer-KFGame.ini           | % { (get-content $_.fullname -raw ) -replace "(?ms)\[IniVersion\](.*)\r\n\s*\r\n",$KFGame | Set-Content -Path $_.fullname}
Get-ChildItem  -Recurse KFWebAdmin.ini                | % { (get-content $_.fullname -raw ) -replace "(?ms)\[IniVersion\](.*)\r\n\s*\r\n",$KFWebAdmin | Set-Content -Path $_.fullname}
Get-ChildItem  -Recurse KFWeb.ini                     | % { (get-content $_.fullname -raw ) -replace "(?ms)\[IniVersion\](.*)\r\n\s*\r\n",$KFWeb | Set-Content -Path $_.fullname}
Get-ChildItem  -Recurse PCServer-KFSystemSettings.ini | % { (get-content $_.fullname -raw ) -replace "(?ms)\[IniVersion\](.*)\r\n\s*\r\n",$KFSystemSettings | Set-Content -Path $_.fullname}
Get-ChildItem  -Recurse KFAI.ini                      | % { (get-content $_.fullname -raw ) -replace "(?ms)\[IniVersion\](.*)\r\n\s*\r\n",$KFAI | Set-Content -Path $_.fullname}

https://stats.mtreadwell.com/

 

[Emenblade]

yeah i ended up going the http: redirect route for my server. got that all setup just the other day, and then last night my server god ddosed.... apparently thats a thing with his game server..

 

[baztheallmighty]

yeah i ended up going the http: redirect route for my server. got that all setup just the other day, and then last night my server god ddosed.... apparently thats a thing with his game server..

See this post for the DDOS issues and solutions. You will need a Linux machine to implement most solutions.

Server - KF2 (or any Unreal Engine 3) server on RedHat/CentOS/Rocky/Alma Linux DDoS defense with the help of firewalld

--list-all is a wrong command you should run the command that displays the content of the network block, e.g.: firewall-cmd --ipset=networkblock --get-entries Also, at this point as an end customer who may not be into all the nuts and bolts I'd rely on Zmey's solution...

forums.tripwireinteractive.com

For the http side of things I would also recommend using a provider aws, azure,etc or if you self host a CDN like cloudflare, Fastly, etc in front of your web server.

 

[Emenblade]

For the http side of things I would also recommend using a provider aws, azure,etc or if you self host a CDN like cloudflare, Fastly, etc in front of your web server.

Yeah I'm already in that forum post, it's kinda over my head to be honest, but I'm trying to get to speed.

As for the http map redirect server, I'm running all my stuff on my NAS in my closet. The map files aren't huge, I figured I'd just upload them to the players from my end. A cdn sounds a bit excessive. I mean if some how I have 6 players downloading the map at the same time that might be a bit slow as I've only got gigabit up. But for me and a bud or two I figured that would be enough. Would the cdn protect my network?
It seems like the only this thus far that's being mistreated is the game server it's self.

 

[baztheallmighty]

You are correct the biggest problem is the DDOS amplification attacks against the game itself. An explanation of a very similar problem with ntp is here https://www.cloudflare.com/learning/ddos/ntp-amplification-ddos-attack/ .

I previously wrote about a possible solution to the problem also:

I disagree, the ddos amplification attacks are relatively easy to solve. One simple way would be to force the client to send more data than the server responds with initially. After the first packet, the connection request, is sent by the client you put a for loop client side that sends say 60 additional packets. These packets contain nothing useful. On the server side you have another while loop waiting for and counting those 60 packets. Once the server has the 60 packets it responds and everything continues like normal. If the server doesn’t get those 60 packets it timeouts and drops the connection without responding.
Note:I just chose 60 because it’s two seconds at the default server tick rate of 30 ticks per second. The number of packets would need to be modified and packet loss would need to be taken into account. Maybe the server would wait until it has 30 packets then proceed.

This solution would make using kf2 servers for amplification attacks useless because the client has to send more packets to the server then it replies with. The only drawback I see is it would take a little longer to connect. I suspect this would be negligible.

However as you said this is a money issue. The code to do something like this would be relatively simple however there would need to be a large amount of testing after which is what would cost lots.

source:

Constant DDoS

Is anyone going to address the ongoing issue of ddos amplification attacks? for those of use using cable internet theres no way to mitigate these attacks unless its a ISP level. I shut my server down due to these attacks and would like to know if anyone figured out a real solution. Ive tried all...

forums.tripwireinteractive.com

Regarding cdn's:
A cdn is a good idea also. What a cdn does it proxy all of the requests via there infrastructure. User -> cdn -> your web server. Only the cdn will make requests to your web server, you firewall off the rest of the internet as it doesn't need to access your web server. This reduces your risk profile. I use cloudflare. It's free. This means if someone tries to attack my web server they are effectively attacking Cloudflare. Cloudflare filters the requests before sending them to my server.

 

[Emenblade]

I'll have to look into the CDN!