Where are you filing? I'm filing only the posterior aspect of the screw hole of the grip.
The 3 or 4 pairs of grips that I have installed have all fit perfectly except the screw hole was just a little too far toward the front of the frame--less than 1/16 of an inch. All that I have needed to do was extend the screw hole of the grip just a little bit toward the rear of the pistol, thus allowing the grip screw to pass freely into the screw hole of the frame. As I mentioned previously, I used a small round file, such as a 5/32 inch file. You can also use a small round hobby file.
I believe that Mr. Spegel makes his grips this way purposely. The plan then is that if the screw holes don't line up, then minimal adjustment is needed. Either way the grip should fit quite well into the frame--at least mine have. If yours doesn't, then I'm sure that Mr. Spegel can help you.
Rob