Not only is it OK to use a real phrase like "gentleman's agreement", it communicates better. People will more easily understand what you mean, and they won't be distracted when you attempt to be gender-neutral by calling attention to gender.
The above is all true but only because gender-neutral words are not yet used widely enough, and the only way to change that is to systemically prefer gender-neutral words. Isn't it part of hacker mentality that one should fix broken things and one should not be afraid of change?
(I guess this is largely culture related thing, but e.g. among my peers (high educated 30s Europeans) gender-biased words actually cause more distraction.)