I recently started thinking about this the other way - that it's actually more racist to accuse someone making statements about group differences of attributing them to race - rather than culture.

It's a fact that the way people are raised has a huge influence on how conformist they are as adults; in a conversation about the average level of conformity between two different cultures, someone who comes along and brings race into it is the one who is lowering the tone.

Culture is important and heritable, let's just talk about that!