Ping rises gaming on my LOCAL network server. Can someone EXPLAIN the package routes?

[Utopia-Phoenix]

^^topic

I have my primary gaming PC as R7-1700 and GTX 1050 ti which is attached DIRECTLY to the router (Phicomm K3). I have a server PC (Celeron G1610T) attached to the router via a GBE switch (Netgear GS605 V4). All connections are wired. (There are other wireless devices connected to the router, but omitted)

Just a few hours ago I and my dudes were gaming in rooms on this server PC, and all of us suffered a ping rise. (Note: It's a stable ping rise instead of a spike) To my experience, when I game in my own server my ping should be around 8-12. However it rose to as high as 28 in that moment, and astonished me.
I want no more of this happen again to my gamers and tried to find out wtf is happening, but it turns out to be quite clueless. It could either be the network stability problem of the ISP (China Unicom Tsingtao) or down to my trunk router (Phicomm K3) or the server PC itself (G1610T+8GB DDR3).

Now there is a question. Which is the bottleneck of such ping rising gaming experience?

The ISP? But I also suffered the ping rise, and ISP seems unlikely to affact that unless the packets travel through WAN. While gaming KF2 on local servers, does my network packets go through the WAN port? Does ISP instability affact the gaming experience of my OWN?

The Server PC itself? The server has a Celeron G1610T CPU and 8GB DDR3 memory. It used to be running 5 server colsoles.
Recently someone reported that he's suffering ping rises those days, and I shut one of them down. Now it only carries 4 server consoles. I asked the same guy and he replied that it's much better.
Is it the time to close another server console? Or do I have to upgrade my CPU or RAM? I reckon not because I've just closed one console recently.
Also I have another one to run the server colsoles. A Notebook aka Dell N4110. It has i7-2670qm CPU which is far better than the G1610T. If CPU is the bottleneck then it'll tell.

The last and worst case is the ROUTER. The Phicomm K3 is one of the few GBE routers I could lay hand on, and it has BEASTLY wireless intensity. (8 antennas, 4X4 MU-MIMO respectively for 2.4G and 5G) If this router has instability issues then I'd have to make a difficult decision between let alone or change it completely.

 

[Foster Parent]

What is your disk IO situation on the server?

 

[Utopia-Phoenix]

What is your disk IO situation on the server?

Very interesting you mentioned this. It's a Samsung XP941 M2 PCIE SSD! As fast as 1080/450 MB/s reading/writing, I reckon this outperforms 99% of homebuilt servers!

This SSD has little load at all and I reckon it to bear a 10% occupation at most.

 

[Foster Parent]

That is fast. Maybe it is the Celeron. I will ask someone more technical.

 

[Utopia-Phoenix]

That is fast. Maybe it is the Celeron. I will ask someone more technical.

I've attached both server PCs to the router- The Celeron and The Dell N4110. That's 8 servers in total. And there are 8 port forwarding rules on it redirecting accesses to either server PC.

Appendix: Detailed server specifications
(Both servers ran Windows 10 professional)

The Celeron:
Intel Celeron G1610T at 2.3 Ghz, 2/2 core/threads
MSI B75A-G41
8GB DDR3 1600Mhz Crucial ballistix sport, single channel
Samsung XP941 M2 PCIE SSD, 128 GB, at 2.0 x4 bandwidth, connected to PCH PCIE of B75 platform, system drive
(Note: The MB BIOS had FFS insertion to support booting via PCIE SSD devices under UEFI)
NIC: Realtek RTL8111E GBE (1Gbps)

Dell N4110 Notebook:
Intel Core i7-2670qm at 2.8 Ghz under all core loads, 4/8 core/threads (Note that Intel Core series CPU has higher clockrates under monothread load)
Dell N4110 MB HM67
8GB DDR3 1600Mhz Samsung, single channel
Sandisk X300S SATA SSD, 256 GB (System drive)
NIC: Realtek RTL810x/8139 (100Mbps)

 

[®omano]

Usually if ping rises, it is a sign that you are overloading CPU. Monitor you CPU usage because I guess your CPU is just barely handling the game servers. If you don't even know what is going on by actually watching it, you can ask yourself and others all questions this will not give you the answer, so go monitor your resources and then see if you overload your machine. Note that you have only two cores (and not the most powerful out there) to run 4 or 5 servers, so if all game servers are used at same time you may completely overload your CPU. After monitoring CPU usage you'll see.

Did you try with the other i7 computer or did you just post the specs for the sake of it?

 

[Foster Parent]

The responses from the team tend to suspect the Celeron. UR3+ likes 3 or more cores/threads for graphics, game loop and audio. WWise likes even more performance for audio. "It could be the game loop just running out of steam on a dual-core box and upping it to quad might be the answer" and "the Phincomm seems like a pretty capable box".

Another possibility mentioned was, packets going over the MTU size barrier. So you could "try increasing the MTU on the switch/server/client to well above 1500, like 3k or some such"

 

[®omano]

UR3+ likes 3 or more cores/threads for graphics, game loop and audio. WWise likes even more performance for audio.

what are you talking about, he's having problem with SERVER, he runs the client on a powerful computer. (and UR3+ like in Unreal Rengine 3+ ?)

 

[Foster Parent]

what are you talking about, he's having problem with SERVER, he runs the client on a powerful computer. (and UR3+ like in Unreal Rengine 3+ ?)

KF2 is built with a modified version of Unreal. UR3 was a typo - should have been UE3.
Yoshiro said, in 2013: "The base engine uses 2 main threads (and thus cores) and will spin off other processes to a third thread. Some of our work will have it creating more threads to spin off..."
The KF2 server software shares a lot code with the client. I know people set it to run on single cores. I've also seen it run fine on a fast i3 with 2 cores (and virtual threading) but a little more CPU is rarely a bad thing.

I also notice, too late in this case, that Utopia-Phoenix said the 8GB of memory in the server is all in one stick. So, no dual channel action?

 

[®omano]

What I mean is that the server does not use graphics, sound, and so on. Also server is mostly single threaded it is different from client which uses all cores when running.

 

[Utopia-Phoenix]

Usually if ping rises, it is a sign that you are overloading CPU. Monitor you CPU usage because I guess your CPU is just barely handling the game servers. If you don't even know what is going on by actually watching it, you can ask yourself and others all questions this will not give you the answer, so go monitor your resources and then see if you overload your machine. Note that you have only two cores (and not the most powerful out there) to run 4 or 5 servers, so if all game servers are used at same time you may completely overload your CPU. After monitoring CPU usage you'll see.

Did you try with the other i7 computer or did you just post the specs for the sake of it?

I established server consoles on BOTH PCs, The Celeron and the Dell N4110.
I havn't been able to monitor the CPU status at the EXACT moment of sh1tty gaming but I looked into it afterwards. It's about 40% loaded. All 4 rooms are occupied.(Not with 6 men though)
I've seen higher load before. Back to the days of running 5 rooms I'd seen a 67% load and it just worked fine, no ping rises. And there was no such maps known as krampus lair by then...We were playing krampus lair when ping rise happened.

I'd like to power on both server PCs and fill them with players to test if the CPU is the problem. However there is hardly possible. It's difficult to find enough players to fill all 8 rooms.

 

[Utopia-Phoenix]

The responses from the team tend to suspect the Celeron. UR3+ likes 3 or more cores/threads for graphics, game loop and audio. WWise likes even more performance for audio. "It could be the game loop just running out of steam on a dual-core box and upping it to quad might be the answer" and "the Phincomm seems like a pretty capable box".

Another possibility mentioned was, packets going over the MTU size barrier. So you could "try increasing the MTU on the switch/server/client to well above 1500, like 3k or some such"

Well, the Internet settings. In the router settings it said the mtu value is 1480. I reckon that this is the default settings for pppoe connections.

Also, the memory. You are right. I picked the single channel 8GB because it has more hedge ratio (the biggest capacity RAM strips in each generation are resistant to dropouts) and upgrading flexibility. The channel bandwidth is rarely a problem in consumer scenario. (Not in servers though)

Also I forgot one thing. The crucial memory I've been using is working under 1333 mhz due to the limitation of CPU and MB. It has enhanced spd timings though. I havn't noticed the numbers at 1333 but I think it's a C9 instead of C11 at 1600mhz.

 

[Utopia-Phoenix]

KF2 is built with a modified version of Unreal. UR3 was a typo - should have been UE3.
Yoshiro said, in 2013: "The base engine uses 2 main threads (and thus cores) and will spin off other processes to a third thread. Some of our work will have it creating more threads to spin off..."
The KF2 server software shares a lot code with the client. I know people set it to run on single cores. I've also seen it run fine on a fast i3 with 2 cores (and virtual threading) but a little more CPU is rarely a bad thing.

I also notice, too late in this case, that Utopia-Phoenix said the 8GB of memory in the server is all in one stick. So, no dual channel action?

Hey dude, i fled from work to home to check out the server Celeron and the Dell N4110. I noticed that on the newly rebooted celeron platform, the RAM usage in taskmgr is 4.0/2.8/1.2 GB occupied/backup/available (translation error may occur as original language is Chinese)

However, when I suffered another minor ping rise (from 12-16 to 20) I looked into the celeron server, and it said the RAM usage to be 4.0/3.8/almost zero occupied/backup/available. Is this the reason of ping rises?
At this hour 3 out of 4 rooms are filled (6/6) with the remaining one empty. That comes in a total of 18 men gaming in my server.

This is my luck to have the celeron server filled with men to duplicate this ping rise problem, but not in the N4110 server. Probably I had to try again to see.

 

[Foster Parent]

I wonder about a single stick of RAM. Don't most MB chipsets try to use dual channels for RAM, to improve performance?

One other thing, one of the dev's that would know pointed out that the server will only run at 30FPS, so a change of ping from 0 to 20ms changes nothing.

 

[Utopia-Phoenix]

I wonder about a single stick of RAM. Don't most MB chipsets try to use dual channels for RAM, to improve performance?

One other thing, one of the dev's that would know pointed out that the server will only run at 30FPS, so a change of ping from 0 to 20ms changes nothing.

Yes, the MB is designed to run on dual channel. That's a function back to ancient. I don't know about old hardware but it is already supported on Intel Sandy bridge platforms (which is dated back to 2011). It's me who did not install a pair of rams onto it, for hedge and easier upgrading sake.
(Also, I checked up the celeron server and confirmed that the single channel Crucial DDR3 is running at 1333 Mhz at SPD timings )

The PING change from 0 to 20 ms is nothing important- I can barely agree although it doesn't sound that suspicious. I have to point out that the primary reason of most game delays (PING number) is the NETWORK i.e. it's the time taken to sent the packets all the way around. However in my case that is NOT right. Suffering a ping rise to myself rather than the other gamers via wan can be hardly explained as network issues because it's a LOCAL network!

Also, when the sh1t happened 2 days ago I was gaming with them. Despite the fact that my ping is very low compared to others EVEN IF I suffered a rise (from 8-12 to 28, which is lower than any normal WAN gamers) I myself witnessed troubles such as gorefasts warped from my front, which is a few steps away, to chopping hours in a blink of an eye. i.e. they warped instead of walked.
I think this is called a "packet loss" as KF2 (and all FPS games i guess) is using UDP for communication. It refreshes zed/player locations at 30 Hz (I guess that's the "fps" of the server?). If a packet is lost then forget about it as its data would be replenished really soon. However when a lot of packets are lost then there will be warping problems as their movements are discontinuous.

Packet loss came in 2 reasons- the network and the server. I'm afraid the latter is right.
If ping rises from 8-12 to 28 are due to network then it'll be really small issues because it means nothing but the packet delays are longer. The packets sent by servers are in order. It's like you playing tennis and your opponent moves afar.
If ping rises are due to server then there is trouble. It means that the server has trouble answering ping requests and the packets sent by servers are disturbed. It's like you are too tired to launch a tennis ball at your opponent.

Today I was trying to compare gaming experiences on the Celeron server and the N4110 server. However I failed to duplicate the exact sh1ts on the celeron server (ping rising, swears of my teammates because of deaths due to zed warps,etc). I only suffered from occasional slightly warps of my own steps but that's already something illustrative. As with the N4110 server I managed to get 3 out of the 4 rooms gaming and witnessed a ping rise to 20 on a fierce battle at burningparis. However this number is iffy- it could be either an enhanced result from the celeron server due to the increased performance of the CPU, or due to the lower number of men, (The rooms are not full compared to fullup condition at the celeron server) or as simple as deviation from local network traffic. I had not duplicated zed/player warp problems on the N4110 server but that may be down to the sake of lower men count thus day. That's how I wasted a full night gaming in order to hunt for abnormals only to found nothing, and the results doesn't tell me sh1T!

Actually I can make some guess at this moment. It's probably the RAM capacity incompetent to carry a full load of 4 servers with 6 men each. The CPU is not my first suspect because I've seen heavier loads before. (Back to the days of 5 servers, that Celeron may have 67% load on it upon fully manned and still didn't make warping and ping rise troubles to gamers including my own.) It can't be down to the drives/NICs because the Celeron platform outperforms the notebook in these aspects while still sucks at service. (PCIE SSD vs SATA SSD, GBE vs 100Mbps)

The RAM capacity. Hmmm. I remembered that one year ago, when I first established the server, the 8GB DDR3 could carry 5 rooms with efforts to spare. (My friend as another server host, he recommended 6 instead of 5) Now it seems that he had troubles carrying 4 rooms! That's f**king weird and I'd like to know why it raises so fast!

 

[Utopia-Phoenix]

I wonder about a single stick of RAM. Don't most MB chipsets try to use dual channels for RAM, to improve performance?

One other thing, one of the dev's that would know pointed out that the server will only run at 30FPS, so a change of ping from 0 to 20ms changes nothing.

Dude, does the KF2 consoles use pagefile? Does the disk space of pagefile affact server performance?
I read a document of an internet cafe with the same problem, and it mentioned two things. The pagefile and the NIC settings.
(The NIC part said to set the MASSIVE SENDING SEGMENTATION V2 IPV4 to be true, which is already that by default.)

Pagefile. I looked into my server and it says automatically determined as 1280 MB in disk C. (Yeah, the XP941.)
1280MB is only 1.25 GB and I reckon it too small to do anything, and a set it to MANUALLY determined as a minimum of 4096 MB and a maximum of 8192 MB.

Don't know if it works. It's done just yesterday 2300 hours UTC+8. Less than 24 hours earlier.

 

[taiiat]

since you're observing being near or at capacity on atleast some of the Resources of the Server, those do sound like culprits to look at first.

setting a larger Pagefile could be a good test case here for using that as Memory overflow if the Server is infact hitting Memory limits.
now obviously i wouldn't ask someone to do this for the long run since that's an awesome way to burn out the SSD, but testing for a day or two to see what changes is good science. and if it does infact help, then more Memory for the Server may be a good answer.
but as noted, also keep an eye on the Processor, to try and determine if it is pretty much at capacity as well.
another system with a better Processor (which at a quick look on Intels' data, certainly looks like a much better Processor) is also a good idea to be able to compare to try to get your science done faster. and more reliably, since as we know reading a Graph for CPU Utilization is a very vague tool at best for tasks that don't scale perfectly (so basically all real world tasks).

since the other system also has the same Memory capacity, that can let you use it to rule out Memory Capacity if that system doesn't experience this issue.
it does throw a wrench into the works since it's a 10/100 NIC you say? Bandwidth wise that's still probably fine but i would keep an eye on that in testing to make sure it isn't skewing your test results.


while i'm here i'll mention the potential of thermals? you say the Processor claims it is not close to being at capacity but even so, i would check what it says for Thermals and look to keep them low if possible (i.e. if they're pretty high, when is the last time you dusted that system out, are any of the Fans starting to fail, Et Cetera).

 

[Utopia-Phoenix]

since you're observing being near or at capacity on atleast some of the Resources of the Server, those do sound like culprits to look at first.

setting a larger Pagefile could be a good test case here for using that as Memory overflow if the Server is infact hitting Memory limits.
now obviously i wouldn't ask someone to do this for the long run since that's an awesome way to burn out the SSD, but testing for a day or two to see what changes is good science. and if it does infact help, then more Memory for the Server may be a good answer.
but as noted, also keep an eye on the Processor, to try and determine if it is pretty much at capacity as well.
another system with a better Processor (which at a quick look on Intels' data, certainly looks like a much better Processor) is also a good idea to be able to compare to try to get your science done faster. and more reliably, since as we know reading a Graph for CPU Utilization is a very vague tool at best for tasks that don't scale perfectly (so basically all real world tasks).

since the other system also has the same Memory capacity, that can let you use it to rule out Memory Capacity if that system doesn't experience this issue.
it does throw a wrench into the works since it's a 10/100 NIC you say? Bandwidth wise that's still probably fine but i would keep an eye on that in testing to make sure it isn't skewing your test results.


while i'm here i'll mention the potential of thermals? you say the Processor claims it is not close to being at capacity but even so, i would check what it says for Thermals and look to keep them low if possible (i.e. if they're pretty high, when is the last time you dusted that system out, are any of the Fans starting to fail, Et Cetera).

Processors. That may be the issue but actually both the celeron and the 2670qm are at low to medium utilization under fully manned right now. I think it's around 50% for the celeron (in taskmgr's graph) and dim for the i7. I've seen higher loads before (at the days of 5 rooms in the celeron i've seen a 67%, still no apparent misactions)
One of my pals said that he knew a server servicing personnel from Qihoo 360 Co.Ltd. He said that no servers dare to be fully loaded aka 100% utilized in taskmgr. Is it possible that my celeron is in such case? AKA it's incompetent to handle this job despite it has only 50% loads?

As with the memory and the pagefile I wouldn't expect or want anything with frequently actions to be inside the SSD. Actually I'd like to turn off the pagefile completely if I could. But no, turning it off may lead to mulfunctions on some applications. Don't know if KF2 servers has such issues.

The primary problem in server comparison of celeron and the notebook is that I can hardly find enough gamers to fill one server (that's 24 men- 6 men* 4 rooms), let alone both. Thus testing became extremely difficult.

The NIC for the celeron platform is GBE, and the notebook is 10/100Mbps. That should be nothing bandwidth wise. (5 rooms, when all manned, use about 1.5Mbps upload and 800Kbps download.)

Thermals? Well... The G1610T is DELIDED with liquid metal dude! Even under a FULL load (100% Utilization, when I was installing softwares and patches) its temperature hardly breaches 50C/122F. Also this server is FANLESS. I have fans but I've detached all of them since I consider them making more heat and noise with their powers then the system itself.