The biggest problem is not so much the performance (although I do feel I should be getting more). It's the jerkiness and changes to the FPS which make the game feel disjointed and stuttery, almost like lag.
For instance, my FPS on medium settings is usually about 40-60FPS. Which would be fine. But it seems everytime I go through a door, or pop up from cover, or bring up the ironsights, or do anything where the scene might change substantially, the game kind of 'stutters' slightly, i guess due to the FPS dropping a lot for a brief moment. This is the key problem, the game doesn't feel smooth, each transition feels laggy to me. It might only be a stutter of a split second but its enough to be a huge annoyance.
A good example of this is the PPSH. I find that when I bring the ironsights up and tap fire, there is a slight stutter. Then the game kinda catches up, and the barrel has jerked up. This was annoying the other day on Station when I fired at a crouching German maybe 25m away - I was sure I must of hit him, but I couldnt be sure because of the slight stutter. The result was that instead of being riddled with bullets, he returned fire with one shot from a K98 and killed me. Things like that, when little perfoamnce hitches cause you to get killed, are what makes for a lot of the frustration with performance, rather than actual low FPS counts across the board.