Governance, procurement and the frontline: seven takeaways on humanitarian AI from Geneva
CDAC Network and Access Now convened a panel during Geneva’s digital week to ask a question the sector can no longer defer: how AI should be governed once it is already in use.
Hosted by the Geneva Centre of Humanitarian Studies at the University of Geneva, the panel brought together three perspectives rarely in the same room: Access Now’s research on how technology is bought and controlled, the SAFE AI Framework’s operational approach to AI governance, and the frontline experience of a Global South organisation delivering information to crisis-affected communities.
The panel – Giulio Coppi (Access Now), Michael Tjalve (Humanitarian AI Advisory and the SAFE AI team) and Stella Suge (FilmAid Kenya), moderated by Ila Schoop Rutten (CDAC) with opening remarks from Valérie Gorin (Geneva Centre of Humanitarian Studies) – surfaced seven themes that should shape how the sector thinks about AI over the next year.
1. The risk isn’t hypothetical
Much of the sector’s AI use is unapproved. Staff reach for public tools to translate, assess needs and manage information because approved alternatives do not yet exist. This is the sector’s ‘shadow AI’ problem, and Michael Tjalve put it directly: people find value in these tools whether or not headquarters sanctions them, and denying that does not remove the exposure. Guilio Coppi was blunt on the consequences: the usual assumption is that weak AI governance might cause harm in the future; his assessment, drawn from Access Now’s research, was that ‘bad things have already happened’ and the sector is ‘very bad at tracking’ them.
His examples shared included a rule-based helpline chatbot converted into an AI system through a routine vendor update, forcing the organisation to shut the service down, and AI entering asylum-processing systems through updates rather than public tenders, among others. Vendor behaviour and a technology’s purpose can change rapidly once an organisation becomes dependent on it.
2. Procurement is the sector’s biggest opportunity
Access Now’s central contribution, drawn from Coppi’s report Reinventing humanitarian aid procurement for the age of AI, was that AI rarely enters an organisation through a deliberate decision. It arrives through a software update, a cloud contract renewal, or a feature switched on by default. Contracts, licensing terms and vendor conditions are often inaccessible even to the teams meant to govern them, and a small number of cloud providers control much of the digital infrastructure the sector relies on. Coppi’s proposal is to treat technology procurement officers as strategic advisers who track vendor ownership, geopolitical exposure, military ties and long-term risk alongside cost and compliance (as the sector already does for critical assets such as vehicle fleets.)
3. SAFE AI turns principles into decisions
The panel was clear that oversight and protection from bias and surveillance come from better decisions rather than better models, and that most responsible-AI frameworks are too generic for humanitarian use. SAFE AI, the Standards and Assurance Framework for Ethical AI, is built to close that gap, helping teams ‘go from abstract responsible AI principles’ to concrete steps. It organises a project across four stages, from problem definition to deployment and monitoring, and applies three risk tiers so that scrutiny matches the stakes. Its readiness assessments and decision gates make stopping a legitimate outcome, so deciding not to use AI counts as responsible practice rather than a failure of innovation.
Developed by CDAC Network with The Alan Turing Institute and Humanitarian AI Advisory, with founding investment from the FCDO and consultation with more than 50 practitioners and community members, it is the foundation for assurance architecture for the sector rather than an ethics checklist.
4. The frontline case: more need, less money, and languages the tools cannot read
Stella Suge grounded the discussion in FilmAid Kenya’s work communicating and translating information into 19 languages for communities from across the Horn of Africa in refugee camps in Kenya. As funding falls and needs rise, many local organisations experience AI as a necessity rather than an option, hoping it helps with translation, prediction and reaching more people with less. Her warning was that efficiency cannot override safety, dignity and accountability, and that the languages her teams work in are poorly supported by commercial tools. A system that cannot recognise people’s languages and contexts cannot represent their needs.
5. Communities have a right to know, and organisations have a duty to know
Affected people’s ‘right to know’ when AI shapes decisions about them ran throughout the panel’s discussions. The panel pushed back on the idea that digital literacy is something only affected communities lack. In fact, humanitarian staff often can’t say where their own data is stored or who profits from it. Understanding the full data lifecycle – what a tool does, who controls it and how it might fail – is part of the duty to do no harm. Alongside communities’ right to know, Coppi advised that agencies have an ‘obligation to know’ as part of our sector obligation to do no harm.
6. Dependency, lock-in and the equity of localisation
New tech risks creating new dependencies. Large organisations are often locked into legacy systems that are difficult and expensive to leave. Smaller, local organisations may be more agile, but they rarely get recognised – or funded – as serious digital actors. Rather than simply rejecting vendors, the panel suggested building in protections from the start, including: exportable data, replaceable systems, opt-in (not opt-out) AI features, and published contracts and stacks that the rest of the sector can learn from. If headquarters keep secure corporate systems while local actors are left with inferior free tools, AI will deepen the inequalities the sector says it wants to close.
7. The sector needs to talk about what didn’t work
Failed pilots, paused projects and tools rejected by communities rarely get documented, so the same mistakes get repeated across the sector. Saying no to AI or stopping partway through when appropriate should not be seen as failure of innovation. Under SAFE AI’s decision gate model, it is a valid and sometimes necessary outcome – as Tjalve put it, ‘it’s not a failure, it’s a very successful outcome’ for the right context.
SAFE AI – the Standards and Assurance Framework for Ethical AI – turns governance principles into operational decisions at each stage of an AI project, from problem definition through design and development to deployment and monitoring. The fragments of good practice already exist across procurement, frontline delivery and framework design. SAFE AI is built to connect them into operational decisions and disclosable, comparable products (such as the SAFE AI Transparency Card) that a country office, a procurement officer and an affected community can all recognise.