Well a lot depends on the finish in question. All holsters will wear the finish but kydex wears it different than leather. Leather will wear on the high points and kydex will wear on any point that contacts the pistol. There is a thread in the Ed Brown forum now with pics of how kydex wears a Gen III finish.
I have a TRP with the SA Armory Koat and kydex has had little effect on the finish. It does ride most of the time in leather but I have carried it in a Comp Tac hybrid MTAC.
if you are interested in the hybrid style. There is a company, and maybe more than one that makes a leather lined version of a hybrid. I have one and love it. It is by the same people making the Silent Thunder holsters that folks on here review and really seem to like.
Mine is just getting better with use and time. It was very stiff at first, but now wraps and fits snugly to the body and even makes the pistol more concealable.
The only complaint I have at all is that it is not easy on and off. I have to pull mine regularly, and that is a bit of a pain.