The grammar of interactive explanatory model analysis.

The growing need for in-depth analysis of predictive models leads to a series of new methods for explaining their local and global properties. Which of these methods is the best? It turns out that this is an ill-posed question. One cannot sufficiently explain a black-box machine learning model using a single method that gives only one perspective. Isolated explanations are prone to misunderstanding, leading to wrong or simplistic reasoning. This problem is known as the Rashomon effect and refers to diverse, even contradictory, interpretations of the same phenomenon. Surprisingly, most methods developed for explainable and responsible machine learning focus on a single-aspect of the model behavior. In contrast, we showcase the problem of explainability as an interactive and sequential analysis of a model. This paper proposes how different Explanatory Model Analysis (EMA) methods complement each other and discusses why it is essential to juxtapose them. The introduced process of Interactive EMA (IEMA) derives from the algorithmic side of explainable machine learning and aims to embrace ideas developed in cognitive sciences. We formalize the grammar of IEMA to describe human-model interaction. It is implemented in a widely used human-centered open-source software framework that adopts interactivity, customizability and automation as its main traits. We conduct a user study to evaluate the usefulness of IEMA, which indicates that an interactive sequential analysis of a model may increase the accuracy and confidence of human decision making.

Intra-Company Crowdsensing: Datafication with Human-in-the-Loop.

Every day employees learn about things happening in their company. This includes plain facts witnessed while on the job, related or not to one's job responsibilities. Many of these facts, which we call "occurrence data", are known by employees but remain unknown to the company. We suppose that some of them are valuable and may improve the company's situational awareness. In the spirit of mobile crowdsensing, we propose intra-company crowdsensing (ICC), a method of "extracting" occurrence data from employees. In ICC, an employee occasionally responds to sensing requests, each about one plain fact. We elaborate the concept of ICC, proposing a model of human-system interaction, a system architecture, and an organizational process. We position ICC with respect to related concepts from information technology, and we look at it from selected organizational and managerial viewpoints. Finally, we conducted a survey, in which we presented the concept of ICC to employees of different companies and asked for their evaluation. Respondents positive about ICC outnumbered skeptics by a wide margin. The survey also revealed some concerns, mostly related to ICC being perceived as another employee surveillance tool. However, useful and acceptable sensing requests are likely to be found in every organization.