Ya, the voice is REALLY bad in this game. It's so quiet and crackly. Every now and then someone with a good mic who is speaking loudly can be somewhat audible.
The biggest complaint I have about the game is the voice audio. Second to that is the in-game audio (money money money, I'm trying to heal you, etc). Not a big issue while playing during a wave, but WOA is it ever annoying after. Wish it was "If soundBite played within the last X seconds, don't play"
Ventrillo seems to be the easiest solution to in game VOIP though.