Design Considerations for Moderation Tooling
Ensuring protection of the protectors.
Overview of thoughful and protective design of tooling for moderating user-generated content, placing emphasis on minimising the psychological effects of exposure to heinous content, while balancing efficiency, accuracy, and the long-term wellbeing of trust and safety teams. Covering techniques for the mitigation of impact where applicable.
https://vale.rocks/posts/moderation-tooling-design
One of the scariest experiences of my life is being in my friend’s Toyota Starlet. It was an abused old thing, adorned with crude stickers (‘Objects in the rear-view mirror are about to get chopped’ and ‘Gas or ass, no free rides’), a mostly attached driver-side door, and a wing spoiler. The springs were aggressively chopped such that it rode extremely low.
We, that is the driver, another friend, and I, decided to venture to a shopping centre. I sat in the front next to the driver, and the other friend curled up in something between a foetal and brace position across the back seats.
Getting in the car proved a lapse in judgement from the second the shitbox began swerving through the car park and lurched onto a busy main road. With a flagrant disregard for the speed limit, it staggered along to find a position in traffic before coming over the crest of a hill and rocketing down it. The tottering Starlet weaved across lanes and through traffic. Traffic which, thanks to the chopped springs, loomed over us and threatened to simply crush us when merging.
As we reached the shopping centre, he cut across a few lanes of traffic before, without slowing down, hurling us off the road into a notoriously tight car park. As the car attempted to hurdle over the speed bumps, the concrete and asphalt reached up and gripped the undercarriage with scraping claws. The grating of metal reverberated awfully through the interior. The driver wound his window down and feathered the pedals as he slowly progressed, screaming profanities and moaning in sympathetic pain, most of his body hanging out the window, as he tried to avoid grinding away what was left of the thinning bottom wall of the petrol tank.
Once past the worst of the speed bumps, he proceeded to pull the wrong way into a one-way road that was thankfully uninhabited by other drivers. He sped down it, performed some further illegal manoeuvres, then completely ignored a stop sign. When he pulled into a park (across two bays, naturally), I took a look at the back seat and my friend, who had all but perfected the contortionist-style ball she’d arranged herself into.
We clambered out of the vehicle, alive, and spent a brief period in the shopping centre before realising our break was nearly spent and that we’d be late to class if we walked back. The driver gleefully hopped back in, and the other passenger and I reluctantly returned to our positions in the car.
Knowing what to expect and with our sphincters suitably puckered, the Starlet threw itself back onto the street. The hill we’d zipped down previously (I think the brakes were shot) was now a mountain for the strained little car to manage. We started off strong with some momentum, but as it reached towards the apex, it slowed, and the tortured engine groaned ominously. The car just made it over the crest, screaming, before plummeting down the other side and back to college in time for class.
