05/07/2024 | News release | Distributed by Public on 05/07/2024 05:20
On 25 March 2024, the Department for Science, Innovation and Technology (DSIT) published new guidance on procuring and deploying AI technology and systems responsibly in the HR and recruitment sector. The guidance is aimed at a non-technical audience.
Developed with feedback and contributions from the ICO, EHRC, Ada Lovelace Institute, APSCo UK, Autistics, CIPD and REC, this guidance warns that though AI-enabled tools are great assets to the recruitment process, these technologies also expose companies to novel ethical risks. To help combat them, it highlights various assurance mechanisms that can be used at each stage of procurement and deployment. These mechanisms can provide organisations with the tools, processes and metrics to evaluate the performance of AI systems, manage risks and ensure compliance with statutory and regulatory requirements.
The guidance further highlights that the procurement, deployment and use of AI should adhere to the UK government's five regulatory principles:
The guidance emphasises the risks that come with the use of AI in recruitment - the biggest of these being unfair bias and discrimination against applicants, particularly those whose age, disability, socioeconomic status or religion mean they may not be proficient in, or have access to, technology. While AI has great potential, its use should be balanced against the risk of digital exclusion of these groups of individuals. Work should be done to combat this risk.
An article in the Financial Times entitled "For young people, the job search has never been so miserable" by Margaret Heffernan gives an insight into some of the problems of relying on AI to sift candidates in the recruitment process. It refers to a 2021 study by Harvard Business School that revealed 90% of employers use automated tracking software to sift through applications despite most acknowledging that these systems vet out qualified candidates simply because they do not match the precise criteria in the job description. With automated systems also not often giving feedback, many potentially good candidates are being missed at early stages of the recruitment process.
To help organisations minimise these risks, the guide contains an annex with a non-exhaustive list of AI tools used in recruitment processes and their associated problems.
The guide identifies various assurance mechanisms that should be used at each stage of the procurement and deployment process, and provides examples and links to third party resources of templates for each of them.
Such mechanisms include:
The mechanism(s) needed will vary depending on which stage of procurement or deployment of the AI system an organisation is at. The guide breaks down which of these may be helpful at each stage with some useful examples.
Alongside this, the guide also explains what organisations should consider at each stage of procurement and deployment of AI systems.
Prior to procuring any AI system, organisations should consider their desired purposes and output and whether the system's functionality can produce them.
Consideration must also be given prior to procurement to applicant accessibility requirements and any reasonable adjustments potentially required to ensure applicants with protected characteristics are not disadvantaged. Failure to do this may lead to organisations being found to have breached the Equality Act 2010. The guidance additionally encourages organisations to consider whether their AI system will involve solely automated decision-making and, if so, whether a data protection impact assessment should be completed prior to procurement.
Moving on to the stage of seeking tenders to supply an AI system, organisations will need to consider the accuracy of the system, any risks that accompany it, and its performance and capabilities. Transparency again is key here to ensure organisations properly alert candidates to any risks with the system that arise from these considerations.
Before deploying a third-party AI system, the guide recommends that organisations run a pilot with potential users. One objective of such a pilot should be to ensure that the employees who will be using the system come away from the pilot clear on the purpose, functionality and outputs of the system, and that feedback is collected from them on any initial issues.
Further, an assessment should be carried out against equality outcomes to identify any learnt bias or inaccuracies produced by the system towards groups with protected characteristics under the Equality Act 2010. Not only will this allow any such bias to be addressed early on, but it will also help to identify any reasonable adjustments that may need to be made for applicants with protected characteristics so that these may be implemented before the system is rolled out in full.
Once the AI system is fully rolled out, ongoing monitoring will be necessary to ensure any errors or bugs that reduce the system's effectiveness can be fixed. User feedback should be continuously sought to help with this.