City to Regulate Use of Artificial Intelligence in Hiring

Regulating the robots. New York City regulates everything from gas pipe and facade inspections to building carbon emissions. Now it’s going to regulate robots. Starting Jan. 1, 2023, the city will prohibit employers and employment agencies from using certain artificial intelligence (“AI”) screening tools in the hiring or promotion process until the tools have been subject to a “bias audit.” While few co-op or condo boards themselves may currently be using AI in their hiring process, it is a growing trend and likely to be used by their vendors, such as management companies, contractors, law firms and accounting companies. As such, it’s something boards need to be aware of.

Getting rid of the human element. Local Law 144 governs Automated Employment Decision Tools, or AEDTS, which include “any computational process, derived from machine learning, statistical modeling, data analytics or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for… employment decisions.” In basic terms, it applies to AI modeling or data processing used to screen or rate candidates for a position based on a score, classification or ranking. Examples include resume scanners that prioritize applications using certain keywords, chatbots that ask job candidates about their qualifications and reject those who do not meet pre-defined requirements, and testing software that provides “job fit” scores for applicants regarding their personalities, aptitudes or cognitive skills.

(Like what you're reading? To get Habitat newsletters sent to your inbox for free, click here.)

The bias audit requirement. Employers and employment agencies will be prohibited from using AEDTS unless two criteria are met. First, the AEDT must be subject to a “bias audit” within one year of use. Then, prior to the use of the AEDT, a summary of the bias audit and the distribution date must be posted to the employer or the employment agency’s website. Any such bias audit must be conducted by an independent auditor (a person or entity not involved in using the AEDT).

There are two types of bias audits. When an AEDT selects individuals to move forward in the hiring process or classifies them into groups, the bias audit must calculate the selection rate for each category and calculate the impact ratio for each category. The categories are race, ethnicity and gender. Where the AEDT scores individuals only, the bias audit must calculate the average score for individuals in each category and the impact ratio for each category.  

The bias audit must then be posted on the employer or employment agency’s website, and it must remain posted for at least six months after the AEDT was last used.

Giving notice. Any employer or employment agency that uses an AEDT must notify candidates who reside in New York City that an AEDT will be used as well as the job qualifications and characteristics that are being considered. The notice must be provided at least 10 business days before the use of the AEDT. It must also include instructions for how to request an alternative selection process or accommodation. 

If the notice is not on a website, it must be provided within 30 days of a written request from the applicant. The employer or the employment agency must provide instructions for how to make a written request if the information is not posted on the website.

Penalties for failure to comply. An employer or employment agency that fails to comply with this new law is subject to monetary penalties: up to $500 for the first violation and each additional violation occurring on the same day as the first violation. Each subsequent violation will cost an employer between $500 and $1,500.  

Each day that an AEDT is used in violation of the law will give rise to a separate violation. Any failure to provide notice to a candidate of the use of an AEDT is a separate violation with a separate monetary penalty.

Andrew I. Bart is an attorney at the law firm Borah Goldstein Altschuler Nahins & Goidel.