Say you want to adopt a dog, from a picture, and you task your machine learning system to classify the image as either a husky, which would be safe to adopt, or a wolf, which probably is not a good idea. Can you get that photograph classified with certainty?  “Because researchers don’t have insights into what is going on they can easily be misled,” said Sameer Singh, assistant professor in the UCI Department of Computer Science. “Classification is core to machine learning,” said Singh, describing ‘black box’ machine learning predictions at the Association for Computing Machinery (ACM) July 12 meeting at the Cove. Machine learning is pervasive in our lives—from email to games. “It’s in our phones,” said Singh, a machine learning  and natural language processing expert. “It is in our houses. It is basically everywhere.”One of his students created a wolf/dog classifier in a few hours that seemed to work—at first.

“How can we trust that the predictions are correct?” Singh asked. How can we trust that the machine learning algorithm is doing what we think it is doing? How can we understand and predict the behavior? How can we improve it to correct potential mistakes?” As an illustration, he showed that the seemingly accurate classifier his student had created was in fact just learning how to recognize snow in images: if it detected snow, it predicted wolf, and if not, it predicted husky; definitely not very intelligent behavior.

Singh provided a brief background of the research in machine learning, which, for a long time, consisted of much simpler classifiers like linear models and simple trees. “We can look at the structure of these classifiers and know whether we can trust them or not,” Singh said. “The structure will tell us what happens with the data, or show where the error is and how to fix it. But looking at structure is beneficial only when structure is simple.”

Big data makes everything complicated

According to Singh, when big data arrived a decade or so ago, it exploded the number of applications. Big data means more complexity, encompassing many more complex codes in multiple dimensions and complex surfaces. Essentially, big data turned machine learning algorithms into “black boxes.” According to Singh, the more accurate the algorithm, the harder it is to interpret, especially for deep learning.

The LIME Approach—Explaining decisions of multiple algorithms

Singh introduced the LIME algorithm, a model-agnostic approach that can explain decisions made by any machine learning classifier. Since LIME treats the classifier as a black box, it can interpret any algorithm’s decision without sacrificing its accuracy. The main idea behind LIME is not to explain the whole surface, but to zoom in to the local behavior of a data point. This, for example, allows LIME to determine what parts of the image that are useful for the classifier to predict. i.e., his student’s classifier is only looking at the ground to differentiate between a wolf and a husky. The rest of the picture doesn’t matter.

Trust and verify

His group designed a quantitative evaluation to decide if the explanations generated by LIME are useful or not, based on four criteria:

  • Can we understand what the machine learning algorithms are doing?
  • Can we compare different machine learning algorithms?
  • Can we improve the existing machine learning algorithms?
  • And can we predict how machine learning will behave on new data?

The evaluation, carried out by crowd-sourced testers from Amazon’s Mechanical Turk service, demonstrated how explanations can help non-experts to understand machine learning algorithms. For example, users without any explanations only got about 60 % accuracy in predicting what the machine learning algorithm would do. However, when given explanations, users got over 90% accuracy with less time and effort. It took them between four to six seconds rather than 12 seconds to predict what the machine learning model would do on a certain data point. “Explanations are important,” Singh concluded.