Algorithmic systems and the presumption of innocence: legal analysis of biases and protection mechanisms
The article examines the impact of modern artificial intelligence technologies on the fundamental right of an individual to be presumed innocent until a court judgment becomes final and legally binding. The analysis establishes how automated facial recognition, algorithmic risk-assessment tools for recidivism, and predictive policing systems can introduce bias due to flawed data sampling, opaque algorithmic assumptions, and imperfect modeling methodologies. It finds that deploying these technologies without adequate oversight threatens to shift the burden of proof and to violate the “in dubio pro reo” principle enshrined in Article 6 of the European Convention on Human Rights. The study analyzes Directive (EU) 2016/343 and Regulation (EU) 2024/1689 (the AI Act), which establish minimum standards for criminal proceedings and set requirements for the transparency and accountability of algorithmic systems, and it reviews European Parliament resolutions and recommendations from Fair Trials and Amnesty International concerning defense access to source code and algorithmic audit results. Based on the identified risks, the article argues for the introduction of explainability mechanisms, the creation of independent AI audit bodies, and legislative restrictions on the autonomous use of high-risk technologies without human involvement. It also emphasizes the need for specialized training of judges, prosecutors, and defense attorneys in artificial intelligence and algorithmic fairness, as well as for guaranteeing the accused’s right to review expert assessments of the algorithmic tools used as evidence. It is established that the defense should have a statutory right to access the technical documentation of an algorithm (including descriptions of the sources of training data, validation methods, and the results of independent audits), while duly taking into account regimes for protecting trade secrets. The importance of providing mechanisms for confidential in-court review and of state-funded expert examinations where an individual cannot secure such review independently is analyzed. The advisability of imposing a procedural prohibition on the use of fully autonomous decisions in matters that directly restrict an individual’s liberty (for example, grounds for arrest or for extending a preventive measure) is substantiated – in such cases, the decision must be made by a human decision-maker who is required to consider the explanation provided by the algorithm and to record the reasons for accepting or rejecting its conclusions. It is recommended to introduce, at the national level, supervisory and certification procedures for high-risk algorithms, including periodic independent audits and public reports on their effectiveness and on any biases detected. In the context of transnational electronic evidence, emphasis is placed on the need to take into account the practices of the SIRIUS and TREIO projects when harmonizing rules on data access and cooperation with foreign jurisdictions, in order to prevent algorithmic evidence obtained abroad from evading proper scrutiny.
Paper
Full text
Algorithmic systems and the presumption of innocence: legal analysis of biases and protection mechanisms
Semantic Scholar · 2025
Abstract
The article examines the impact of modern artificial intelligence technologies on the fundamental right of an individual to be presumed innocent until a court judgment becomes final and legally binding. The analysis establishes how automated facial recognition, algorithmic risk-assessment tools for recidivism, and predictive policing systems can introduce bias due to flawed data sampling, opaque algorithmic assumptions, and imperfect modeling methodologies. It finds that deploying these technologies without adequate oversight threatens to shift the burden of proof and to violate the “in dubio pro reo” principle enshrined in Article 6 of the European Convention on Human Rights. The study analyzes Directive (EU) 2016/343 and Regulation (EU) 2024/1689 (the AI Act), which establish minimum standards for criminal proceedings and set requirements for the transparency and accountability of algorithmic systems, and it reviews European Parliament resolutions and recommendations from Fair Trials and Amnesty International concerning defense access to source code and algorithmic audit results. Based on the identified risks, the article argues for the introduction of explainability mechanisms, the creation of independent AI audit bodies, and legislative restrictions on the autonomous use of high-risk technologies without human involvement. It also emphasizes the need for specialized training of judges, prosecutors, and defense attorneys in artificial intelligence and algorithmic fairness, as well as for guaranteeing the accused’s right to review expert assessments of the algorithmic tools used as evidence. It is established that the defense should have a statutory right to access the technical documentation of an algorithm (including descriptions of the sources of training data, validation methods, and the results of independent audits), while duly taking into account regimes for protecting trade secrets. The importance of providing mechanisms for confidential in-court review and of state-funded expert examinations where an individual cannot secure such review independently is analyzed. The advisability of imposing a procedural prohibition on the use of fully autonomous decisions in matters that directly restrict an individual’s liberty (for example, grounds for arrest or for extending a preventive measure) is substantiated – in such cases, the decision must be made by a human decision-maker who is required to consider the explanation provided by the algorithm and to record the reasons for accepting or rejecting its conclusions. It is recommended to introduce, at the national level, supervisory and certification procedures for high-risk algorithms, including periodic independent audits and public reports on their effectiveness and on any biases detected. In the context of transnational electronic evidence, emphasis is placed on the need to take into account the practices of the SIRIUS and TREIO projects when harmonizing rules on data access and cooperation with foreign jurisdictions, in order to prevent algorithmic evidence obtained abroad from evading proper scrutiny.