Right. Read that first. Also the Santa Clara Principles that Doctorow mentions.[1]
Now, a key point there is freedom from arbitrary action. The Santa Clara Principles have a "due process clause". They call for an appeal mechanism, although not external oversight. Plus statistics and transparency, so the level of moderation activity is publicly known, to keep the moderation system honest.
That's really the important part. The moderation process is usually rather low-quality, because it's done either by dumb automated systems or people in outsourced call centers. So a correction mechanism is essential.
It's failure to correct such errors that get companies mentioned on HN, in those "Google cancelled my account for - what?"
The "Abuse prevention is tradecraft" author has hold of the wrong end of the problem.
Note that Facebook has the Oversight Board to handle appeals and I assume such appeals must necessarily reveal the underlying decision making process. https://www.oversightboard.com/
> I assume such appeals must necessarily reveal the underlying decision making process.
Probably not the parts they keep secret. The Oversight Board can make a decision about content based on the content itself and publicly-available context.
What tells the automated system that flagged it initially used don't need to be revealed, and the feedback from the Oversight Board probably isn't "make these detailed changes to the abuse detector algorithm" but a more generalized "don't remove this kind of stuff".
Now, a key point there is freedom from arbitrary action. The Santa Clara Principles have a "due process clause". They call for an appeal mechanism, although not external oversight. Plus statistics and transparency, so the level of moderation activity is publicly known, to keep the moderation system honest.
That's really the important part. The moderation process is usually rather low-quality, because it's done either by dumb automated systems or people in outsourced call centers. So a correction mechanism is essential.
It's failure to correct such errors that get companies mentioned on HN, in those "Google cancelled my account for - what?"
The "Abuse prevention is tradecraft" author has hold of the wrong end of the problem.
[1] https://santaclaraprinciples.org/