You have just found one of the largest advantages of an XSE over the "sought after" XS. (I prefer the original style safety on the XSE as well.) Yes, the XSE uses the standard Colt style rear sight (not the huge XS sight) and thus can be fitted with a BoMar by any competent pistolsmith. I am not a machinist, but I believe that by moving the sight slightly back from it's standard "Series 70" location, you miss the firing pin lock plunger hole completely with the dovetail cutter. It is done everyday - I just don't have the specs in front of me this early in the morning.
Do a search under "BoMar" and I think you will find the proper fore/aft location for the BoMar cut on any standard Colt slide. It has been a topic on this Forum. Come to think of it, I believe Gold Cups and XS guns are the only ones that can't be easily fitted with a BoMar. A truly great sight. Colt is factory fitting them to Gold Cups and some special runs of 1991A1s.
If you have no luck, let me know. I'll ask our shop machinist for you.