The padlock symbol never goes away even after the rank is attained.
On a server with realistic loadouts disabled, you'll have a choice between the default pistol (Nagant revolver or P38) or the unlocked pistol (TT33 or C96). With the default setting, a rifleman gets neither option, and so you get no reward for rank 2 beyond the insignificantly small boosts to a handful of stats.
Only the commander, marksman, machine gunner and AT soldier always have pistols.