Explainable Decisions of Algorithms Using Examples | Rutgers University Innovation Ventures

Explainable Decisions of Algorithms Using Examples



Artificial Intelligence algorithms (AI) and Machine Learning (ML) are used across a widening array of application domains. To trust the decisions made by these algorithms it is important for people to understand how they arrive at them.

Invention Summary:

Researchers at Rutgers University have created a novel solution that uses Bayesian Teaching for explaining the decisions made by AI algorithms.  Bayesian Teaching is a model-agnostic system that samples data subsets to explain model inferences to a domain (but not necessarily technical) expert.

This solution allows a user to ask questions of a ML model (used by an AI Algorithm to make a decision) and in response it provides (e.g., displays) examples to explain the reasons for the decisions.  For a use case, consider a user who wants to know why an AI algorithm in a self-driving automobile made a specific decision; our solution provides responses such as: “in similar situations it was found that the reason for this decision were: “example reason 1”, “example reason 2”, etc.  The examples might be in the form of displayed images (see above) or some other form relevant to the type of application/use.

Advantages:

  • Use of bayesian teaching ensures that the best sets of examples are generated according to a model.
  • Supports any type of machine learning model that has probabilistic interpretation (e.g., supervised, unsupervised and reinforcement learning (including deep learning)) models. 

Applications:

This invention can be used in any functional area where explanations of decisions of AI/ML algorithms are desired or necessary. Examples are:

  • Enables the enforcement of the requirements of European Union General Data Protection Regulation (GDPR) compliance.
  • Provides the ability to determine whether machine learning models cause decisions that discriminate based on race and ethnicity (bias detection). For example, to determine if there is bias in AI/ML applications that perform bank loan and mortgage processing, resume processing for matching candidates/positions, and targeted advertising.

Intellectual Property & Development Status:

The technology is patent pending and is currently available for licensing.

Patent Information: