From Vice:
Scientists Increasingly Can’t Explain How AI Works
AI researchers are warning developers to focus more on how and why a system produces certain results than the fact that the system can accurately and rapidly produce them.
By Chloe Xiang
November 1, 2022, 6:00am… Most AI systems are black box models, which are systems that are viewed only in terms of their inputs and outputs. Scientists do not attempt to decipher the “black box,” or the opaque processes that the system undertakes, as long as they receive the outputs they are looking for. …
These types of AI systems notoriously have issues because the data they are trained on are often inherently biased, mimicking the racial and gender biases that exist within our society.
… Black box models can be extremely powerful, which is how many scientists and companies justify sacrificing explainability for accuracy.
… “The risks are that the system may be making decisions using values we disagree with, such as biased (e.g. racist or sexist) decisions. Another risk is that the system may be making a very bad decision, but we cannot intervene because we do not understand its reasoning,” Jeff Clune, an Associate Professor of Computer Science at the University of British Columbia, told Motherboard.
… Predictive AI systems can also guess a person’s race based on X-rays and CT scans, but scientists have no idea why or how this is the case.
As we all know, Race Does Not Exist Biologically so the fact that AI can pretty accurately guess a patient’s race from scans, even though human radiologists can’t (or at least won’t admit to in print), is proof that AI is in league with Devil.
… At the same time, some experts argue that simply shifting to open and interpretable AI models—while allowing greater visibility of these processes—would result in systems that are less effective.
“There are many tasks right now where black box approaches are far and away better than interpretable models,” Clune said. “The gap can be large enough that black box models are the only option, assuming that an application is not feasible unless the capabilities of the model are good enough. Even in cases where the interpretable models are more competitive, there is usually a tradeoff between capability and interpretability. People are working on closing that gap, but I suspect it will remain for the foreseeable future, and potentially will always exist.”
… In recent years there’s been a small but real push by some in the industry to develop “white-box models,” which are more transparent and whose outputs can be better explained (it’s worth mentioning that the white-box / black-box terminology is in itself part of a long history of racially coded terms in science; researchers have pushed to change “blacklist” to “blocklist,” for example.) …
A more challenging problem is that many AI systems are designed for the concept of universalism—the idea that “[a] system is good if it works everywhere for everyone at all times” Keyes explained. “But the problem is that that’s not how reality works, different people are going to need different explanations of different things.
As you have been explained to over and over, Reality = My Truth Is My Lived Experience (for certain people, but not for you).
If we really want AI to be more explainable, we actually have to really, fundamentally change how we imagine and how developers imagine AI.”
But maybe humans could do a better job of explaining AI pattern recognition if humans stopped lying to themselves about real patterns so much?
… Debiasing the datasets that AI systems are trained on is near impossible in a society whose Internet reflects inherent, continuous human bias. Besides using smaller datasets, in which developers can have more control in deciding what appears in them, experts say a solution is to design with bias in mind, rather than feign impartiality.
“The approach I currently think is the best is to have the system learn to do what we want it to,” Clune said. “That means it tries to do what we ask it to (for example, generate pictures of CEOs) and if it does not do what we like (e.g. generating all white males), we give it negative feedback and it learns and tries again. We repeat that process until it is doing something we approve of (e.g. returning a set of pictures of CEOs that represents the diversity in the world we want to reflect).
In other words, white and Indian male tech CEOs wants their corporations’ AI to generate pictures of a nonexistent world with lots of black women CEOs without themselves giving up their jobs to black women. Win-win, everybody!
This is called ‘reinforcement learning through human feedback’,
Dog-training for AI
because the system is effectively using trial and error learning to bring its outputs in line with our values. It is far from perfect, and much more research and innovation is required to improve things.”
As we continue to negotiate where and how AI should be used, there are many things to consider before we start letting AI hire people or decide who to give loans to.
As we saw during the Bush Era Housing Bubble, the latest human thinking about whom to give loans to is practically infallible.
