NYC Publishes Final Rule on Use of Automated Employment Decision Tools

April 26, 2023 Advisory

Earlier this month, the New York City Department of Consumer and Worker Protection published its Final Rule implementing provisions of the New York City Administrative Code (the Code) relating to the use of Automated Employment Decision Tools (AEDTs). Section 20-870 of the Code defines an AEDT as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision-making for making employment decisions that impact natural persons.”

The Final Rule clarifies the definition of AEDTs in the Code by explaining that “machine learning, statistical modeling, data analytics, or artificial intelligence” that fall within the definition of an AEDT include “mathematical, computer-based techniques” that generate a prediction (e.g., likelihood of success) or categorization (e.g., skill set or aptitude) for which a computer identifies the inputs, their relative importance and other parameters designed to improve the accuracy of the prediction or classification.

Section 20-871(a) of the Code prohibits employers from using an AEDT unless the AEDT has been subjected to a bias audit within the past year, the employer posts a summary of the audit results on its website, and notice has been provided to employees and job candidates. The Final Rule aids employers in complying with the Code by providing more information about what the bias audit must include. Among other things, the audit must calculate the “impact ratio” – a comparison of the selection rate or scoring rate for a given category to the most selected or highest scoring category – for sex, race/ethnicity, and intersectional categories of sex, ethnicity and race. A category that comprises less than 2% of the data being used can be excluded from the impact ratio calculations. However, the audit must indicate the number of individuals the AEDT assessed but were not included in the audit calculations because they were placed in an unknown category.

The Final Rule provides that generally, an employer’s bias audit must be based on its own historical data (i.e., data from past uses of the AEDT). However, an employer can rely on a bias audit of an AEDT using the historical data of other employers or employment agencies if the employer provided its own historical data to the independent auditor conducting the audit or has never before used an AEDT. Alternatively, if insufficient historical data is available, the employer can use test data. If it does so, the summary of the audit must explain why historical data was not used and how the test data was obtained.

While New York City’s efforts to prevent bias in the use of AEDTs are laudable, it remains to be seen if they will be sufficient, or to what extent they may result in unintended consequences for employers and employees alike. Neither the Code nor the Final Rule explains when or how historical or test data must be collected. This is significant, because audits will only be accurate if the historical data or test data is representative of the pool of people from which the employer is choosing candidates or employees. If the data is not representative, or if the employer has changed its hiring practices or selection criteria since the historical or test data was collected, the differences or changes may create sources of bias not reflected in the audit. Additionally, as artificial intelligence models are used and develop over time, they sometimes perform in ways that users did not instruct them to and do not expect. If an AEDT changes the manner in which it operates or is further developed after an audit, the changes and developments may introduce new sources of bias. While the Code attempts to anticipate these problems by requiring audits to be conducted annually, given the rapid progress of artificial intelligence models to date, even this may not be enough to avoid pitfalls that may be associated with the use of AEDTs. 

Before enforcement of the AEDT provisions of the Code and the Final Rule begins, employers should determine whether they use AEDTs in making their employment decisions and, if so, identify an appropriate independent auditor and provide the auditor with historical data or test data to conduct an audit. It will also be important to review the license terms or the Software as a Service terms for the AEDT to ensure the auditor will deliver a product or service that will enable the organization to comply with the Code and Final Rule. Employers should also consider requesting access to any similar audits that the auditor has conducted. Finally, employers should keep in mind that these are not the only rules guiding how businesses in New York City may use AEDTs. The federal Equal Employment Opportunity Commission published guidance in 2022, in connection with its Algorithmic Fairness Initiative, on the potential discriminatory impact of such tools from the perspective of its enforcement of the Americans with Disabilities Act.

If you have questions about how to comply with the AEDT provisions of the Code and Final Rule, reach out to the authors listed below or your regular Armstrong Teasdale lawyer. Our Employment and Labor practice will continue to monitor developments regarding the Code, Final Rule and other efforts to regulate AEDTs.

Contact Us
  • Worldwide
  • Boston, MA
  • Chicago, IL
  • Denver, CO
  • Dublin, Ireland
  • Edwardsville, IL
  • Jefferson City, MO
  • Kansas City, MO
  • Las Vegas, NV
  • London, England
  • Miami, FL
  • New York, NY
  • Orange County, CA
  • Philadelphia, PA
  • Princeton, NJ
  • Salt Lake City, UT
  • St. Louis, MO
  • Washington, D.C.
  • Wilmington, DE
Worldwide
abstract image of world map
Boston, MA
800 Boylston St.
30th Floor
Boston, MA 02199
Google Maps
Boston, Massachusetts
Chicago, IL
100 North Riverside Plaza
Suite 1500
Chicago, IL 60606-1520
Google Maps
Chicago, Illinois
Denver, CO
4643 S. Ulster St.
Suite 800
Denver, CO 80237
Google Maps
Denver, Colorado
Dublin, Ireland
Fitzwilliam Hall, Fitzwilliam Place
Dublin 2, Ireland
Google Maps
Edwardsville, IL
115 N. Second St.
Edwardsville, IL 62025
Google Maps
Edwardsville, Illinois
Jefferson City, MO
101 E. High St.
First Floor
Jefferson City, MO 65101
Google Maps
Jefferson City, Missouri
Kansas City, MO
2345 Grand Blvd.
Suite 1500
Kansas City, MO 64108
Google Maps
Kansas City, Missouri
Las Vegas, NV
7160 Rafael Rivera Way
Suite 320
Las Vegas, NV 89113
Google Maps
Las Vegas, Nevada
London, England
Royal College of Surgeons of England
38-43 Lincoln’s Inn Fields
London, WC2A 3PE
Google Maps
Miami, FL
355 Alhambra Circle
Suite 1200
Coral Gables, FL 33134
Google Maps
Photo of Miami, Florida
New York, NY
7 Times Square, 44th Floor
New York, NY 10036
Google Maps
New York City skyline
Orange County, CA
19800 MacArthur Boulevard
Suite 300
Irvine, CA 92612
Google Maps
Philadelphia, PA
2005 Market Street
29th Floor, One Commerce Square
Philadelphia, PA 19103
Google Maps
Philadelphia, Pennsylvania
Princeton, NJ
100 Overlook Center
Second Floor
Princeton, NJ 08540
Google Maps
Princeton, New Jersey
Salt Lake City, UT
222 South Main St.
Suite 1830
Salt Lake City, UT 84101
Google Maps
Salt Lake City, Utah
St. Louis, MO
7700 Forsyth Blvd.
Suite 1800
St. Louis, MO 63105
Google Maps
St. Louis, Missouri
Washington, D.C.
1717 Pennsylvania Avenue NW
Suite 400
Washington, DC 20006
Google Maps
Photo of Washington, D.C. with the Capitol in the foreground and Washington Monument in the background.
Wilmington, DE
1007 North Market Street
Wilmington, DE 19801
Google Maps
Wilmington, Delaware