Imagine a citizen applying for a service and being rejected because of an automated assessment, a young applicant being excluded from a job opportunity before anyone reads their qualifications, or a transaction being delayed because a system classifies the applicant as high risk.
These hypothetical examples illustrate what can happen when algorithms become involved in decisions affecting people’s lives.
In such situations, an institution cannot simply say, “The system made the decision.”
The questions are: Who established the rules? What data informed the assessment? And who can review and correct the outcome?
In my previous article on artificial intelligence in Iraq, I discussed the importance of responsible and secure adoption. Here, we examine a more specific responsibility: What happens when a technical recommendation becomes a decision affecting a right, an opportunity, or access to a service?
When a Technical Output Becomes a Consequential Decision
Not every automated process involves artificial intelligence. A rejection may result from a fixed programming rule, a statistical model, or a machine learning system.
However, citizens experience the consequences regardless of the technology involved.
There is a difference between a system that organizes files and one that assesses eligibility for financial assistance, supports a medical decision, screens job applicants, or flags suspicious activity.
The greater the consequences of a decision, the stronger the need for testing, oversight, and safeguards. Having an employee click an approval button is insufficient if they accept the output without understanding or reviewing it.
The Algorithm Cannot Bear Responsibility on Its Own
Errors may arise from inaccurate data, poor design, use beyond the intended purpose, or ignored warnings.
The responsibilities of the deploying institution, developer, data provider, and supervisory authority must therefore be defined according to their roles, contracts, and applicable law.
An institution adopting a system should not make the supplier the citizen’s only point of contact for complaints. Equally, a developer should not automatically escape responsibility after concealing a defect or making misleading claims.
UNESCO’s Recommendation on the Ethics of Artificial Intelligence emphasizes that ultimate responsibility must remain attributable to individuals or legal entities. This is an international guiding principle; it does not replace the need to establish legal responsibility in each case.
Historical Data Can Reproduce Injustice
An algorithm is not neutral simply because it operates through numbers.
If a recruitment system learns from unfair historical decisions, it may reproduce them. If data about a region or social group are incomplete, its results may be less accurate for those affected.
Apparently ordinary information, such as an address or a gap in employment, may also act as an indirect indicator of social circumstances that should not automatically become grounds for exclusion.
A high overall accuracy rate is therefore insufficient. Institutions should examine errors across different groups, the consequences of wrongful rejection, and the system’s ability to handle local names, languages, dialects, and contexts.
Citizens Should Be Able to Understand What Happened
Citizens do not need to read source code, but they need an understandable explanation of decisions affecting them.
Was a document missing? Were the records outdated? Was the outcome based on a probability estimate? Was incorrect information used?
A meaningful explanation identifies the main factors, the limitations of the result, and the appeal process without exposing other people’s data or revealing details that could facilitate fraud.
Model complexity and commercial confidentiality should not become excuses for preventing accountability. If an appropriate explanation cannot be provided for a highly sensitive use, a simpler system or restrictions on deployment may be necessary.
Human Review Must Be Able to Correct the Outcome
Saying “There is human oversight” is not enough.
Reviewers need training, time, information, and genuine authority to change decisions. They must also be able to disagree with the system without pressure to accept its recommendations automatically.
Institutions should provide a clear appeal channel, a published response timeframe, opportunities to correct data, and urgent measures when delays threaten medical treatment or someone’s livelihood.
The Council of Europe’s Framework Convention on Artificial Intelligence addresses documentation, challenges to decisions, and complaint mechanisms. It offers a comparative reference; these safeguards do not automatically become enforceable in Iraq through the convention itself.
Privacy Comes Before Deployment
An institution’s possession of citizens’ data for one purpose does not authorize every subsequent use.
Before training a model or connecting databases, the institution should establish the purpose and legal basis, minimize the information required, and control access and retention periods.
Health, financial, biometric, and children’s data require stronger protection. External services also require scrutiny: Where are the data processed? Are they used for training? Who can access them?
This connects directly with my articles on protecting Iraqis’ data and digital sovereignty. Accountability weakens when institutions cannot trace what happens to their data or audit the systems they depend on.
System Security Is Part of Fair Decision-Making
A harmful decision may result from a deliberate attack rather than a statistical error.
Data manipulation, compromised accounts, and misuse of access privileges can alter outcomes. Generative systems introduce additional risks, including fabricated information and malicious instructions embedded in documents.
Sensitive applications therefore require independent verification, protected logs, security testing, and a mechanism for stopping the system and returning to alternative procedures.
The model version, relevant data, and employee interventions should be documented to the extent necessary for review while protecting privacy. Accountability becomes impossible when we cannot reconstruct what happened.
What Does Iraq Need to Build?
In my view, the practical starting point is an inventory of institutional uses, classification of their risks, and impact assessments before sensitive systems are purchased and deployed.
Procurement contracts should include audit rights, incident reporting, controls on data use, update management, and the ability to move to another provider.
We also need oversight combining technical, legal, and sector expertise, a role for universities in independent testing, and participation by affected groups in evaluating services.
Certain uses that disproportionately threaten rights may require restrictions or prohibition.
Passing an initial test is insufficient because data, circumstances, and models change. Performance, complaints, and differences between groups should be monitored, with reassessment after updates or changes in purpose.
Assistance and alternative channels should also be available for older people, persons with disabilities, and anyone unable to use the platform, so that appealing a decision does not itself become a digital privilege.
Trust Is Built Through the Ability to Correct Mistakes
Artificial intelligence can improve services, detect errors, and expand opportunities when supported by responsible design and effective oversight.
But speed does not justify denying people an explanation of decisions affecting them, and statistical accuracy does not eliminate an individual’s need for a fair assessment.
A trustworthy digital state can answer its citizens: Why was this decision made? Who reviewed it? And how will we correct it if we were wrong?
When these answers are clear, algorithms can serve justice, while institutions remain accountable to people.
Eng. Saja Albayati
Cybersecurity and Digital Transformation Consultant
These hypothetical examples illustrate what can happen when algorithms become involved in decisions affecting people’s lives.
In such situations, an institution cannot simply say, “The system made the decision.”
The questions are: Who established the rules? What data informed the assessment? And who can review and correct the outcome?
In my previous article on artificial intelligence in Iraq, I discussed the importance of responsible and secure adoption. Here, we examine a more specific responsibility: What happens when a technical recommendation becomes a decision affecting a right, an opportunity, or access to a service?
When a Technical Output Becomes a Consequential Decision
Not every automated process involves artificial intelligence. A rejection may result from a fixed programming rule, a statistical model, or a machine learning system.
However, citizens experience the consequences regardless of the technology involved.
There is a difference between a system that organizes files and one that assesses eligibility for financial assistance, supports a medical decision, screens job applicants, or flags suspicious activity.
The greater the consequences of a decision, the stronger the need for testing, oversight, and safeguards. Having an employee click an approval button is insufficient if they accept the output without understanding or reviewing it.
The Algorithm Cannot Bear Responsibility on Its Own
Errors may arise from inaccurate data, poor design, use beyond the intended purpose, or ignored warnings.
The responsibilities of the deploying institution, developer, data provider, and supervisory authority must therefore be defined according to their roles, contracts, and applicable law.
An institution adopting a system should not make the supplier the citizen’s only point of contact for complaints. Equally, a developer should not automatically escape responsibility after concealing a defect or making misleading claims.
UNESCO’s Recommendation on the Ethics of Artificial Intelligence emphasizes that ultimate responsibility must remain attributable to individuals or legal entities. This is an international guiding principle; it does not replace the need to establish legal responsibility in each case.
Historical Data Can Reproduce Injustice
An algorithm is not neutral simply because it operates through numbers.
If a recruitment system learns from unfair historical decisions, it may reproduce them. If data about a region or social group are incomplete, its results may be less accurate for those affected.
Apparently ordinary information, such as an address or a gap in employment, may also act as an indirect indicator of social circumstances that should not automatically become grounds for exclusion.
A high overall accuracy rate is therefore insufficient. Institutions should examine errors across different groups, the consequences of wrongful rejection, and the system’s ability to handle local names, languages, dialects, and contexts.
Citizens Should Be Able to Understand What Happened
Citizens do not need to read source code, but they need an understandable explanation of decisions affecting them.
Was a document missing? Were the records outdated? Was the outcome based on a probability estimate? Was incorrect information used?
A meaningful explanation identifies the main factors, the limitations of the result, and the appeal process without exposing other people’s data or revealing details that could facilitate fraud.
Model complexity and commercial confidentiality should not become excuses for preventing accountability. If an appropriate explanation cannot be provided for a highly sensitive use, a simpler system or restrictions on deployment may be necessary.
Human Review Must Be Able to Correct the Outcome
Saying “There is human oversight” is not enough.
Reviewers need training, time, information, and genuine authority to change decisions. They must also be able to disagree with the system without pressure to accept its recommendations automatically.
Institutions should provide a clear appeal channel, a published response timeframe, opportunities to correct data, and urgent measures when delays threaten medical treatment or someone’s livelihood.
The Council of Europe’s Framework Convention on Artificial Intelligence addresses documentation, challenges to decisions, and complaint mechanisms. It offers a comparative reference; these safeguards do not automatically become enforceable in Iraq through the convention itself.
Privacy Comes Before Deployment
An institution’s possession of citizens’ data for one purpose does not authorize every subsequent use.
Before training a model or connecting databases, the institution should establish the purpose and legal basis, minimize the information required, and control access and retention periods.
Health, financial, biometric, and children’s data require stronger protection. External services also require scrutiny: Where are the data processed? Are they used for training? Who can access them?
This connects directly with my articles on protecting Iraqis’ data and digital sovereignty. Accountability weakens when institutions cannot trace what happens to their data or audit the systems they depend on.
System Security Is Part of Fair Decision-Making
A harmful decision may result from a deliberate attack rather than a statistical error.
Data manipulation, compromised accounts, and misuse of access privileges can alter outcomes. Generative systems introduce additional risks, including fabricated information and malicious instructions embedded in documents.
Sensitive applications therefore require independent verification, protected logs, security testing, and a mechanism for stopping the system and returning to alternative procedures.
The model version, relevant data, and employee interventions should be documented to the extent necessary for review while protecting privacy. Accountability becomes impossible when we cannot reconstruct what happened.
What Does Iraq Need to Build?
In my view, the practical starting point is an inventory of institutional uses, classification of their risks, and impact assessments before sensitive systems are purchased and deployed.
Procurement contracts should include audit rights, incident reporting, controls on data use, update management, and the ability to move to another provider.
We also need oversight combining technical, legal, and sector expertise, a role for universities in independent testing, and participation by affected groups in evaluating services.
Certain uses that disproportionately threaten rights may require restrictions or prohibition.
Passing an initial test is insufficient because data, circumstances, and models change. Performance, complaints, and differences between groups should be monitored, with reassessment after updates or changes in purpose.
Assistance and alternative channels should also be available for older people, persons with disabilities, and anyone unable to use the platform, so that appealing a decision does not itself become a digital privilege.
Trust Is Built Through the Ability to Correct Mistakes
Artificial intelligence can improve services, detect errors, and expand opportunities when supported by responsible design and effective oversight.
But speed does not justify denying people an explanation of decisions affecting them, and statistical accuracy does not eliminate an individual’s need for a fair assessment.
A trustworthy digital state can answer its citizens: Why was this decision made? Who reviewed it? And how will we correct it if we were wrong?
When these answers are clear, algorithms can serve justice, while institutions remain accountable to people.
Eng. Saja Albayati
Cybersecurity and Digital Transformation Consultant
