I think you got it right,you need a fast processor rather than more cores.
QUOTE:
Server Requirements (Beta)
16 Players: Haven't benchmarked this yet, but we don't expect it will take much CPU to run this
32 Players: CPU usage - 1 Core of a 2.6 GHZ Core I7 or equivalent (i.e. the server process will take most of 1 core, with a few smaller threads on other cores). We also tested on an Intel Core2 Quad Q8400 @ 2.66GHz and that could handle 32 players but I wouldn't try and do 32 players with a machine much slower than that.