I always thought the M4 M203 should have manual reload of the grenade instead of automaticly reloading it.
This way, you could fire with the attached grenade launcher, shoot with the assault rifle when needed and reload when it's safe to do so.
But at the moment, the M4 203 grenades is loaded...