Somni Porta
This week’s work was a little lighter on theory but no less deep on implications. From week to week now we’ve been getting into the heart of what I imagined the course would be taking a look at which is the implication of rogue systems. The article we read mentioned the “automated decision systems” I decided to include in my title and the notion really gave me pause for consideration. Safiya Noble and Sarah Roberts weighed in on the topic of “platform capitalism” here, and spoke on the difficulties of a system which is difficult to root out because it exists in the very search system we typically use.
Their argument centered largely around the question of the human component in this machine system. There are people who build these systems, but the AI which is tasked in carrying out the requests and which serves, as we’ve now seen, to partition our results and ourselves from each other does not factor in context. In the article, they mention how Dylann Roof’s search of “black on white crime” resulted in white supremacist results being selected for him to view, and how because the algorithm was so specific to what it viewed he should see, it indeed served to fuel the fire which caused his crimes. There is a serious abdication of “education” here as mentioned by Safiyah on the parts of these developers to the humanities and social sciences in their programming of AI. It’s a tool which could, if it isn’t being misused already at worst, be used to such positive effect, if it’s given the proper care to regulate itself.
As Sarah said, this conversation ought to be taken hold of by us users of this technology and these sites, because otherwise little will change. As she mentions, these companies are no longer merely “tech” companies, but they are in fact “media” companies. At the point where personal data is being used to bring us together or separate us, it creates or dissolves communities of us, and as such becomes quite social.
The social nature of all of this was certainly then felt in the Buolamwini videos. I never know if these sorts of addresses are necessary when discussing such topics, but as a “pale male” as she refers to in the longer-form video, I must assume that mine is not the profile which would have difficulties being “seen” by casual technology. I can say that in my day-to-day, I never use such technology freely. I’ve not held any kind of disdain or mistrust of it, but more and more I’ve realized that no casual usage of these technologies “simply” exists; these things are mined for. In any case, the issue is not simply one of biases in the processing of images of darker skinned peoples, it is exactly what is processed of them that I found troubling. Buolamwini cites that the NYPD uses facial recognition software in ways which are morally and legally questionable, but the point I found the most interesting was the more nuanced of, that these technologies are not appropriately vetted. That is to say, these technologies are utilized for such serious purposes as criminal recognition, but they have such turnovers of false positives and nonrecognition that how can they be allowed to be used at all? And yet, apparently, they are.
Buolamwini also makes the point that to date (at the time of the video at least) that there are no federal regulations of facial recognition technology in the US. I mean, on a personal note, that is frightening. I try not to subscribe to immediate notions of dystopia or mistrust of those who incorporate these technologies for any various purpose, but that there exist no guidelines for what is morally acceptable to do with it is troubling.