your brain can compensate the image of your head and eye moving. However it cannot compensate a moving image on a screen as easily. So I don't think there should be a lot of camera wobble, although perhaps still some to get a nice perception of motion.
A person always sees everything at the same size, rather than only when you are looking through the iron sights while utilizing controlled breathing. Next to that the image in your peripheral vision is at a normal scale as well and not .5 times the normal scale. Next to that you do not either use your peripheral vision or your focus function of the eye, you use both at the same time. It doesn't make fov zoom bad or unrealistic, however it comes with quite some hefty shortcomings.
The issue that I have, is not anything visual like seeing the zooming effect but rather that, to be sure that nobody is looking at me somewhere I need to check it out in zoomed view. Otherwise if I'm not zoomed and the enemy is zoomed he'll be able to see me while I won't see him (when were both equally concealed, still and sharing an equal low profile).
For this I hope that I can zoom at all times, as then I can take a quick look somewhere without having to raise my weapon, especially as you are able to see things in a realistic size at all times in real life. Next to that I hope there is some room for user customization within certain bounds, as different people have different tastes and preferences, different monitors, and distances to their monitors.