A cool experiment in NL: The LegitiMate. Working method to judge the use of algorithms in ADM from 3 perspectives.

Let’s supervise algorithms!

It’s easier said than done. How can a lawyer state that the algorithms execute the law accordingly? How can an IT auditor discover anomalies? And is it possible to audit the quality of algorithms by peer review?

These months we are trying to develop in co-creation a working method to judge the development and use of algorithms, in automated administrative decision making. The project is funded by the Ministry of Internal Affairs.

Our goal is to develop a standardization that as well helps public administration to develop and use algorithms in a responsible and auditable way. And no, we are not interested yet in machine learning algorithms. We recognize that in NL Administrative ADM runs on preprogrammed algorithms, sometimes written in COBOL. That is why we want to shed light on these first.

Together with a group of experts working within the government, academia and the Ombudsman we are trying to analyze and judge the process of translating legislation into code. The Name: The LegitiMate (in Dutch: de LegitiMaat).

Our first findings after one month of research:

  1. Even for experts like ourselves, familiar with the lingo and issues in IT projects, it is very hard to understand how one agency functions regarding the automated execution of the law. Basic words like ‘product owner’ have local meanings and are used for different functions in different agencies. Another issue is that the formal documents not necessarly represent the daily routines and culture. To have a basic understanding researchers need insight in the necessary roles in the production process after which these can be plotted in the organization chart.
  2. There are many bridges to cross in multidisciplinary research. The first bridge is language; even though we are speaking Dutch we experience misunderstandings. We try to deal with this to constantly verify our understanding and use definitions that work for us all.
  3. The businessrules community in public administration is our gateway. They are enthousiastic and helpful and offer us interesting cases for our research.
  4. Our Library of usefull documents / articles is extremely interesting; will post these later!
  5. Auditors are extremely busy around February/March due to their responsability to formally close the previous year.