TalkToModel: Interface for Understanding ML Models

Currently, machine learning models are widely used in various professional fields and form the basis of many mobile applications, software packages and online services. Although many people encounter and interact with these models, few fully understand their operation and underlying processes.

In the modern world of machine learning, models are becoming increasingly complex and feature-rich. Their growth raises an important question: how can we make these models more understandable and interpretable for a wide audience, including specialists without deep knowledge in the field of machine learning?

Researchers from the University of California, Irvine, and Harvard University developed the TalkToModel. It is an interactive conversational system designed to explain machine learning models and their predictions to both professionals and non-expert users. This interface allows you to dialogue with ML models using ordinary natural language.

The research is based on previous developments related to explainable artificial intelligence (XAI) and human-AI interaction. The main goal of this work was to introduce a new platform that could provide clear and accessible explanations of how artificial intelligence works, similar to how OpenAI's conversational platform, ChatGPT, answers questions.

The researchers conducted an experiment involving healthcare workers with varying levels of machine learning experience. Almost all participants were new to the field. They were invited to use TalkToModel to answer questions and understand how machine learning models work.

The results of the study were impressive. Most users preferred to use TalkToModel to understand models. They completed tasks faster and more accurately using this interface. Even machine learning engineers have admitted that TalkToModel is a useful tool.

So how does TalkToModel work? It transforms questions into structured logical forms that allow ML models to provide explanations and interpretations. This approach provides flexibility in dialogue, supporting open inquiry and facilitating understanding of complex models.

TalkToModel is an innovative system that opens the door to natural conversations aimed at understanding machine learning models applied to a variety of tabular datasets and classifiers. Instead of complex programming, users communicate with TalkToModel in natural language (Fig. 1, block 1). The dialog engine parses the input data into the executable representation (Fig. 1, block 2). The execution engine performs the operations, and the dialogue engine uses the results in its response (Fig. 1, block 3).

With TalkToModel, users can discuss why certain predictions occur in a model; the changes that will occur to predictions when input data changes; the ways to change predictions, and more. This analysis can be applied to any group of data, be it one individual instance or an entire category of data.

For example, if you want to predict the development of diseases, you might ask questions like: “How important is body mass index (BMI) for predictions?” or “How will the likelihood of disease change after lowering glucose levels by 10 in men over 20 years old?”. TalkToModel will give you the information, saying that BMI is the most important predictive characteristic, and that decreasing your glucose levels by 10 will reduce your chance of developing diabetes by 20%. After this, you can continue the dialogue by asking additional questions. TalkToModel makes it easier to explain how models work because you can talk to the system in natural language and it will give you informative answers.

You can see an example of such a dialogue in Fig. 2.

To support meaningful conversations with TalkToModel, there are methods to improve language understanding and model explainability. First, a dialog engine is implemented that parses the user text input. This data is converted into a language similar to structured query language using a large language model (LLM). LLM performs parsing by treating the task of translating user utterances into a programming language as a seq2seq learning problem, with user utterances as the source and parsing in the programming language as the target.

Also, the TalkToModel system combines the operations of explanation, machine learning error analysis, data manipulation, and descriptive text generation into a single language that can cover the wide range of potential conversation topics that are needed in most explainable models. Examples of various operations are presented in Fig. 3.

The system offers an operation mechanism that automatically selects the most appropriate explanations and operations for the user. This reduces the burden on users and makes interacting with machine learning models more accessible. In addition, a text-based interface has been created that allows even those without high technical skills to understand and interact with ML models. As a result, TalkToModel makes explaining how machine learning models work more accessible and understandable to a wider audience.

In the future, the use of TalkToModel may expand to include the use of the system in real-life clinical and laboratory settings, where participants can apply it to understand and optimize the performance of models. Also, future research could focus on visualization and analysis of raw data to increase user trust.

TalkToModel is a step forward in the development of the field of explainable artificial intelligence. This interface allows you to talk to complex machine learning models in natural language and understand their decisions. This tool promises to make ML more accessible and interpretable for everyone.

You can find the code of the model on github.