The FP safety plunger should be nice and smooth so it slides easily in the slide. The dents or peening may be caused by the FP hitting the plunger before the series 80 levers push it up into the slide fully and allow the FP to pass.
Your trigger overtravel screw may be set wrong. Your trigger may not be moving back far enough to move the levers enough to push the plunger all the way in. Your FP may be hitting the plunger slightly when you pull the trigger.
Also, is the plunger spring OK? It's not kinked and possibly binding up the plunger?