That is flat out not true. Someone constantly attacking your character to others is going to have an effect. Those other people may be influenced by what that person says, and that will affect how they perceive you, which will in turn affect your involvement in the project.