• Please make sure you are familiar with the forum rules. You can find them here: https://forums.tripwireinteractive.com/index.php?threads/forum-rules.2334636/
  • Weve updated the Tripwire Privacy Notice under our Policies to be clearer about our use of customer information to come in line with the EU General Data Protection Regulation (GDPR) rules that come into force today (25th May 2018). The following are highlights of our changes:


    We've incorporated the relevant concepts from the GDPR including joining the EU and Swiss Privacy Shield framework. We've added explanations for why and how Tripwire processes customer data and the types of data that we process, as well as information about your data protection rights.



    For more information about our privacy practices, please review the new Privacy Policy found here: https://tripwireinteractive.com/#/privacy-notice

Custom Map Install Issue

fwfdf1reman

Member
Aug 11, 2016
145
8
18
66
Joshua, TX.
Well, evidently the issue is NOT fixed.

I slowly started adding maps to only one of our servers to test. So far, I have only loaded 8 of the custom maps we had installed. This morning I get a notice from NFO on ALL of our servers in the "Events Log".

Quoting.................

"The latest version of KF2 causes servers to endlessly re-read workshop content from disk (when workshop usage is enabled), leading to extremely high I/O on the machine hosting the server.

This is an issue that we saw earlier this year when troubleshooting workshop problems with the previous version of KF2. We found that some workshop content would not download, but replacing the Steam libraries with a later version allowed it to download but led to endless I/O.

We have contacted Tripwire to ask them what their ETA is on a resolution. If they are not able to roll one out quickly, we may have to roll back the Steam binaries or disable workshop usage until they can, due to the serious performance impact. "

Was so looking forward to getting the content back on to our servers too.
 

Kittenmittens

Tripwire Interactive Community Team
Staff member
Mar 11, 2016
1,071
271
83
Killing Floor Discord :)
Well, evidently the issue is NOT fixed.

I slowly started adding maps to only one of our servers to test. So far, I have only loaded 8 of the custom maps we had installed. This morning I get a notice from NFO on ALL of our servers in the "Events Log".

Quoting.................

"The latest version of KF2 causes servers to endlessly re-read workshop content from disk (when workshop usage is enabled), leading to extremely high I/O on the machine hosting the server.

This is an issue that we saw earlier this year when troubleshooting workshop problems with the previous version of KF2. We found that some workshop content would not download, but replacing the Steam libraries with a later version allowed it to download but led to endless I/O.

We have contacted Tripwire to ask them what their ETA is on a resolution. If they are not able to roll one out quickly, we may have to roll back the Steam binaries or disable workshop usage until they can, due to the serious performance impact. "

Was so looking forward to getting the content back on to our servers too.
Do you have any information on what way they are going about contacting TWI? When previously discussing this issue with the KF2 team, they asked who NFO had reached out to wanting to see whatever information it was that had been sent.

Yosh and I were referring to the original issue this thread was reporting. We were referring to the outdated files being updated. The high disk usage issue is still being looked at, we got some logs from another server owner.
 

fwfdf1reman

Member
Aug 11, 2016
145
8
18
66
Joshua, TX.
I do no know who NFO reached out to at Tripwire. We have 8 servers, only 1 was being tested to see if the Steam Workshop issue was fixed. However, the high I/O usage was occurring on all 8 of our servers.

The issue was posted on all 8 of our server in the EVENTS log ............... no name is ever attached .............. it's just info for server admins at NFO.

It looked like the one server I was testing was working as far as Custom Maps from the Workshop, BUT, if NFO disables the Steam Workshop because of the high I/O usage,,,,,,,,,,,,, then the fix is mute, right?
 

Kittenmittens

Tripwire Interactive Community Team
Staff member
Mar 11, 2016
1,071
271
83
Killing Floor Discord :)
The high disk usage issue on Windows is like 1+ year old WTF is going on?
This is a different issue. I believe you are referring to one that resulted after some Steam Workshop changes Valve did. The one I kept reaching out for logs over from folks running servers set up in the default manner as laid in the wiki with great deal of custom content and not running servers with maxplayers. No one ever sent us anything like this and we could not reproduce the same reports internally. There was a great deal of discussion on it on our Discord. It was determined by other server owners that for those issues, servers were set up in a way that caused it and if they would adjust it to our recommendations, the issue did not persist.
 

Kittenmittens

Tripwire Interactive Community Team
Staff member
Mar 11, 2016
1,071
271
83
Killing Floor Discord :)
I do no know who NFO reached out to at Tripwire. We have 8 servers, only 1 was being tested to see if the Steam Workshop issue was fixed. However, the high I/O usage was occurring on all 8 of our servers.

The issue was posted on all 8 of our server in the EVENTS log ............... no name is ever attached .............. it's just info for server admins at NFO.

It looked like the one server I was testing was working as far as Custom Maps from the Workshop, BUT, if NFO disables the Steam Workshop because of the high I/O usage,,,,,,,,,,,,, then the fix is mute, right?
No. The needed updated .DLLs were added for your original issue. That is the fix we are referring to here. However hearing from Yoshiro, he is now in touch with someone from NFO and working with them to add to the information we have. Thanks!
 
  • Like
Reactions: fwfdf1reman

®omano

Grizzled Veteran
May 14, 2009
4,061
185
63
France
www.hellsoldiers.tk
This is a different issue. I believe you are referring to one that resulted after some Steam Workshop changes Valve did. The one I kept reaching out for logs over from folks running servers set up in the default manner as laid in the wiki with great deal of custom content and not running servers with maxplayers. No one ever sent us anything like this and we could not reproduce the same reports internally. There was a great deal of discussion on it on our Discord. It was determined by other server owners that for those issues, servers were set up in a way that caused it and if they would adjust it to our recommendations, the issue did not persist.
I understand there are two different issues here:

- map not downloading from workshop at all since early this year, which is fixed since this last update
- high disk usage when using workshop on KF 2 WINDOWS servers

I hear since more than a year, from time to time about this high disk usage on WINDOWS KF 2 servers, you're telling it is because people improperly setup their servers? If so in what way because that surely is an important information I don't see how they could mess up their installation.
 

fwfdf1reman

Member
Aug 11, 2016
145
8
18
66
Joshua, TX.
One of the support guys at NFO responded to me on my ticket that has been open on the Workshop issue, that they contacted Tripwire through the Support Portal on this I/O issue. That's all the info I have.

Also, I do see now that this is a separate issue, but I still maintain that if NFO has to implement their own fix for the I/O issue, then more than likely they will disable the Steam Workshop since it is the constant checking of maps that seems to be causing this. Am I right or wrong on this?
 

®omano

Grizzled Veteran
May 14, 2009
4,061
185
63
France
www.hellsoldiers.tk
I guess there's a bug somewhere in the process, 'consant check' of workshop items does not produce any excessive I/O on my Linux server at all.

It does produce a lot of READ though (maybe this is what is meant by I/O), that's true. I have 15 workshop items and this is what it produces over time look at the DISK READ column):


With multiple servers on the machine, loaded with way more workshop items I can see when it starts to overwhelm the disk.
The Rising Storm 2 servers I host with workshop content do not produce this excessive READ despite having WAY MORE workshop items added to THEM (multiple of them using workshop). So yeah there is definitely something wrong with KF 2 workshop, but I personally don't feel any bad effect, probably because of the 'high end-ness' of the machine I use.

If the issue is the workshop item check every 60 seconds (multiplied by the number of items installed), then maybe, as requested long time ago, allow a server setting to set this checking interval in command line to reduce the disk activity, some people would prefer ONE DAILY CHECK as maps rarely get updated, instead of a check every 60 seconds (or simply a check on server start only).

To me it seems weird that it has to do this excessive READ for workshop, it should check the Binaries/Win64/steamapps/workshop/appworkshop_232090.acf file for the lastupdated/manifest only I do not see why it reads that much it just makes no sense to me, so I guess there is something wrong in your way of checking workshop items status.
 
Last edited:

Kittenmittens

Tripwire Interactive Community Team
Staff member
Mar 11, 2016
1,071
271
83
Killing Floor Discord :)
Replied to you other thread to let you know it was added to the information we have been gathering on this issue. One theory is that it is related to another issue with .u files not pulling from the workshop folder. Thanks
 

fwfdf1reman

Member
Aug 11, 2016
145
8
18
66
Joshua, TX.
I want to thank you both for diving in to this. I am currently still holding off on adding back all the Custom Maps we had on our servers. I feel that it would just be an additional load on them at this time. Thx again for the help.
 

extazy9999

FNG / Fresh Meat
Mar 25, 2018
12
0
1
39
Yup, excessive disk usage problem again... I was asking this issue in 2018 and now it is back. I am tired of this problem.


So far, I am hosting 4 servers that cause 100% ssd usage. It seems checking the custom maps endlessly for nothing. I am thinking to stop hosting my servers as they are not playable due to lag spikes. If TW doesn't fix it, I am done with this game. Thanks.
 

Attachments

Last edited:

®omano

Grizzled Veteran
May 14, 2009
4,061
185
63
France
www.hellsoldiers.tk
Note that this is happening on both Windows and Linux, seems like it may be less intensive on Linux though, I wouldn't have noticed without manually checking.

This should be a high priority hotfix, this is destroying the hardware (most servers now use SSD)..