I thought the whole point of a true refactoring was that it should not lead to any changes in outwardly visible behavior. If you're changing what the code does, that should absolutely be pointed out, regardless of whether it's phrased as "clean up".