I agree. If you have to say "I'm just teasing", you've almost certainly done something wrong.

I'd consider the phrase to be one of those things if you have to say it, you've already messed up. Kind of like "don't worry, he's friendly!"