

Understanding Gradient Rollback
For many, including scientific researchers, artificial intelligence (AI) is a mystery – its reasoning opaque. AI systems and models are often referred to as “black boxes”; we do not understand the logic of what they do. Neural networks are powerful artificial intelligence tools trained to recognize meaningful data relationships and predict new knowledge. Nonetheless, it is not commonly understood how neural networks function or arrive at predictions. When AI systems affect our lives we need to ensure their predictions and decisions are reasonable. NEC Laboratories Europe has recently achieved a milestone in explainable AI research (XAI) by developing the method Gradient Rollback; this opens neural “black box” models and explains their predictions. Gradient Rollback reveals the training data that has the greatest influence on a prediction. Users can ascertain how plausible a prediction is by viewing its explanation (the training instances with the highest influence). The more plausible a prediction is the greater the likelihood that it will be trusted – a key factor in AI user adoption.