Ugh, I had no idea that kind of crap is happening on YouTube.
I agree that we need more humans reviewing things, but the problem is it takes a toll on those humans. I work for a small media company that allows user generated posts and photos. Many of us work from home and only see each other in person once a year or less. Recently at the company retreat I was in a discussion group with the lead content moderator. In my role as an editor I have the ability to flag and remove things, but I don’t have to do it very often as the moderation team takes care of most of it. But I discovered through talking to them I had no idea how much disturbing content gets posted to the site and has to be removed. Everyone feels like our community is safe but they don’t realize what it takes to get it to that point.
The company has a system that automatically flags and/or blacklist certain words, and a system for other users to flag content for review. Thankfully child exploitation content has not been a big problem, but since the site has a lot of mental health support content, they do deal with people posting images of self harm, suicide threats, and sometimes threats of violence towards others. Our lead moderator is one tough cookie, but also has the compassion needed to reach out to the people whose posts are a cry for help. She's amazing and I admire her strength, but there are not many people like her who have what it takes. I'm not sure I could do it. I can deal with nasty comments in text, but photos and video? No way.
Both of the company's current moderators used to work at Huffington Post, and said it was much worse there, but even so, it’s a tough job. My employer treats them well, but most companies don’t pay their moderators well or support their mental health needs, so it’s just creating more psychologically damaged people. I think we need to increase the consequences for people posting the most extreme kinds of content and actually make an effort to track them down. Even though this content is everywhere, I think it’s a relatively small group of people messing up things for everyone. Trolls who have nothing better to do and child abusers who need to be the focus of police stings instead of them hunting down immigrants and low-level drug offenders.