New York S 7599 (Automated decision-making in government agencies)

Summary

Amends the state technology law, education law, and civil service law to regulate automated decision-making by New York government agencies.

Requires government agencies to disclose their use of automated decision-making tools on their websites annually.

Mandates impact assessments for these tools, evaluating their objectives, potential biases, and cybersecurity risks, and ceasing use if they produce discriminatory outcomes.

Instructs agencies to submit impact assessments to the governor and legislature and publish them online.

Establishes protections preventing AI tools from affecting employees' existing rights, roles, or collective bargaining status within educational institutions and public authorities.

Mandates a state inventory of automated decision-making tools, updated annually.

Ensures that usage of AI systems does not result in job displacement or loss of employment benefits.

Preserves existing employment terms and conditions in connection with AI and automated tools.

Amends laws to require government agencies to disclose and assess impacts of automated decision-making tools.

AN ACT to amend the state technology law, the education law and the civil service law, in relation to automated decision-making by government agencies The People of the State of New York, represented in Senate and Assembly, do enact as follows:

Section 1. The state technology law is amended by adding a new article 5 to read as follows:

ARTICLE V AUTOMATED DECISION-MAKING IN GOVERNMENT AGENCIES

Section 501. Definitions. 502. Disclosure of automated decision-making tools by government agencies. 503. Impact assessments. 504. Submission to the governor and legislature.

Defines "automated decision-making tool" using AI, excluding basic software and certain employment tools, affecting individual rights.

§ 501. Definitions. For the purpose of this article:

1. "Automated decision-making tool" shall mean any software that uses algorithms, computational models, or artificial intelligence techniques, or a combination thereof, to automate, support, or replace human decision-making. "Automated decision-making tool" shall not include any software used primarily for basic computerized processes, such as calculators, spellcheck tools, autocorrect functions, spreadsheets, electric communications, or any tool that relates only to internal management affairs such as ordering office supplies or processing payments, and that do not materially affect the rights, liberties, benefits, safety or welfare of any individual within the state. "Automated decision-making tools" shall not include "automated employment decision-making tools" as defined in section four hundred one of this chapter.
2. "Meaningful human review" means review, oversight and control of the automated decision-making process by one or more individuals who understand the risks, limitations, and functionality of, and are trained to use, the automated decision-making tool and who have the authority to intervene or alter the decision under review, including but not limited to the ability to approve, deny, or modify any decision recommended or made by the automated tool.
3. "Government agency" shall mean: (a) the state or civil division thereof; (b) a county, city, town or village; (c) a school district, board of cooperative educational services, vocational education and extension board or a school district as enumerated in section one of chapter five hundred sixty-six of the laws of nineteen hundred sixty-seven, as amended; (d) the state university of New York; (e) the city university of New York; (f) a public improvement or special district including police or fire districts; (g) a public authority, commission or public benefit corporation; or (h) any other public corporation, agency, instrumentality or unit of government which exercises governmental power under the laws of this state.

Requires government agencies to publish and annually update a list of their automated decision-making tools online.

§ 502. Disclosure of automated decision-making tools by government agencies. Any government agency that utilizes an automated decision-making tool, as defined in section five hundred one of this article, shall publish a list of such automated decision-making tools on such govern- ment agency's website no later than the thirtieth of December next succeeding the date on which this section takes effect, and annually thereafter. Such disclosure shall include:

1. a description of the automated decision-making tool utilized by such government agency;
2. the date that the government agency use of such automated decision-making tool began;
3. a summary of the purpose and use of such automated decision-making tool; and
4. any other information deemed relevant by the agency.

Requires government agencies to conduct impact assessments for automated decision-making tools and cease use if discriminatory.

§ 503. Impact assessments. 1. Government agencies seeking to utilize or apply an automated decision-making tool permitted under section five hundred two of this article with continued and operational meaningful human review shall conduct or have conducted an impact assessment substantially completed and bearing the signature of one or more individuals responsible for meaningful human review for the lawful application and use of such automated decision-making tool. Following the first impact assessment, an impact assessment shall be conducted in accordance with this section at least once every two years. An impact assessment shall be conducted prior to any material change to the automated decision-making tool that may change the outcome or effect of such tool. Such impact assessments shall include: (a) a description of the objectives of the automated decision-making tool;

(b) an evaluation of the ability of the automated decision-making tool to achieve its stated objectives;

(c) a description and evaluation of the objectives and development of the automated decision-making including: i) a summary of the underlying algorithms, computational modes, and artificial intelligence tools that are used within the automated decision-making tool; and (ii) the design and training data used to develop the automated decision-making tool process;

(d) testing for: (i) accuracy, fairness, bias and discrimination, and an assessment of whether the use of the automated decision-making tool produces discriminatory results on the basis of a consumer's or a class of consumers' actual or perceived race, color, ethnicity, religion, national origin, sex, gender, gender identity, sexual orientation, familial status, biometric information, lawful source of income, or disability and outlines mitigations for any identified performance differences in outcomes across relevant groups impacted by such use; (ii) any cybersecurity vulnerabilities and privacy risks resulting from the deployment and use of the automated decision-making tool, and the development or existence of safeguards to mitigate the risks; (iii) any public health or safety risks resulting from the deployment and use of the automated decision-making tool; (iv) any reasonably foreseeable misuse of the automated decision-making tool and the development or existence of safeguards against such misuse;

(e) the extent to which the deployment and use of the automated decision-making tool requires input of sensitive and personal data, how that data is used and stored, and any control users may have over their data; and

(f) the notification mechanism or procedure, if any, by which individuals impacted by the utilization of the automated decision-making tool may be notified of the use of such automated decision-making tool and of the individual's personal data, and informed of their rights and options relating to such use.

2. Notwithstanding the provisions of this article or any other law, if an impact assessment finds that the automated decision-making tool produces discriminatory or biased outcomes, the government agency shall cease any utilization, application, or function of such automated decision-making tool, and of any information produced using such tool.

Requires submission of AI impact assessments to government officials and publication on agency websites, with possible redactions.

§ 504. Submission to the governor and legislature. 1. Each impact assessment conducted pursuant to this article shall be submitted to the governor, the temporary president of the senate, and the speaker of the assembly at least thirty days prior to the implementation of the automated decision-making tool that is the subject of such assessment. 2. (a) The impact assessment of an automated decision-making tool shall be published on the website of the relevant government agency.

(b) If the government agency makes a determination that the disclosure of any information required in the impact assessment would result in a substantial negative impact on health or safety of the public, infringe upon the privacy rights of individuals, or significantly impair the government agency's ability to protect its information technology or operational assets, such government agency may redact such information, provided that an explanatory statement on the process by which the government agency made such determination is published along with the redacted impact assessment.

(c) If the impact assessment covers any automated decision-making tool that includes technology that is used to prevent, detect, protect against or respond to security incidents, identity theft, fraud, harassment, malicious or deceptive activities preserve the integrity or security of tools, or to investigate, report or prosecute those responsible for any such malicious or deceptive action, such government agency may redact such information for the purposes of this subdivision, provided that an explanatory statement on the process by which the government agency made such determination is published along with the redacted impact assessment.

Requires the office to maintain and publish an inventory of state automated decision-making tools annually.

§ 2. The state technology law is amended by adding a new section 103-f to read as follows: § 103-f. Automated decision-making tool inventory. 1. The office shall maintain an inventory of state automated decision-making tools. The office shall issue guidance to state agencies identifying the data elements to be collected and submitted to the office for such inventory, including but not limited to the purpose and uses of such automated decision-making tools. The inventory shall be posted on the New York state open data website on the thirtieth of December next succeeding the date on which this section takes effect, and annually thereafter. State agencies shall submit information required by the office at least sixty days in advance of the annual publication date. The office may withhold certain information if it determines disclosure of this information would jeopardize the security of information technology assets, or as prescribed by article six of the public officers law.

2. For purposes of this section, "automated decision-making tool" shall have the same meaning as the term is defined in section five hundred one of this chapter.

3. The office may ask and shall receive from any state agency any information or assistance necessary to carry out its powers and duties under this section.

4. The office shall submit a copy of the artificial intelligence inventory to the governor, the temporary president of the senate, and the speaker of the assembly.

Requires government agencies using automated decision-making tools to disclose detailed use information to the legislature.

§ 3. Disclosure of existing automated decision-making tools. Any government agency, that directly or indirectly, utilizes an automated decision-making tool, as defined in section 501 of the state technology law, shall submit to the legislature a disclosure on the use of such tool, no later than one year after the effective date of this section. Such disclosure shall include: (a) a description of the automated decision-making tool utilized by such agency;

(b) a list of any software vendors related to such automated decision-making tool;

(c) the date that the use of such tool began;

(d) a summary of the purpose and use of such tool, including a description of human decision-making and discretion supported or replaced by the automated decision-making tool; (e) whether any impact assessments for the automated decision-making tool were conducted and the dates and summaries of the results of such assessments where applicable; and (f) any other information deemed relevant by the agency.

Prohibits AI systems from affecting employee rights, positions, duties, and collective bargaining relationships in education.

§ 4. Section 2510 of the education law is amended by adding a new subdivision 4 to read as follows: 4.(a) The use of artificial intelligence systems as defined by section one hundred three-e of the state technology law and automated decision- making tools as defined by section five hundred one of the state technology law shall not affect (i) the existing rights of employees pursuant to an existing collective bargaining agreement, or (ii) the existing representational relationships among employee organizations or the bargaining relationships between the employer and an employee organization.

(b) The use of such artificial intelligence systems and automated decision-making tools shall not result in the: (i) discharge, displacement or loss of position, including partial displacement such as a reduction in the hours of non-overtime work, wages, or employment benefits, or result in the impairment of existing collective bargaining agreements; or (ii) transfer of existing duties and functions currently performed by employees of the state or any agency or public authority thereof to an artificial intelligence system or automated decision-making tool.

(c) The use of such artificial intelligence system or automated decision-making tool shall not alter the rights or benefits, and privileges, including but not limited to terms and conditions of employment and civil service status. The collective bargaining unit membership status of all existing employees of the board of education shall be preserved and protected.

Prohibits AI and automated tools from affecting employee rights, jobs, or collective bargaining agreements.

§ 5. Section 2585 of the education law is amended by adding a new subdivision 5 to read as follows: 5.(a) The use of artificial intelligence systems as defined by section one hundred three-e of the state technology law and automated decision- making tools as defined by section five hundred one of the state technology law shall not affect (i) the existing rights of employees pursuant to an existing collective bargaining agreement, or (ii) the existing representational relationships among employee organizations or the bargaining relationships between the employer and an employee organization.

(b) The use of such artificial intelligence systems and automated decision-making tools shall not result in the: (i) discharge, displacement or loss of position, including partial displacement such as a reduction in the hours of non-overtime work, wages, or employment bene- fits, or result in the impairment of existing collective bargaining agreements; or (ii) transfer of existing duties and functions currently performed by employees of the state or any agency or public authority thereof to an artificial intelligence system or automated decision making tool.

(c) The use of such artificial intelligence system and automated decision-making tool shall not alter the rights or benefits, and privileges, including but not limited to terms and conditions of employment and civil service status. The collective bargaining unit membership status of all existing employees of the board of education shall be preserved and protected.

Prohibits AI systems from affecting employee rights, duties, or bargaining agreements under existing collective agreements.

§ 6. Section 2588 of the education law is amended by adding a new subdivision 9 to read as follows: 9.(a) The use of artificial intelligence systems as defined by section one hundred three-e of the state technology law and automated decision- making tools as defined by section five hundred one of the state technology law shall not affect (i) the existing rights of employees pursuant to an existing collective bargaining agreement, or (ii) the existing representational relationships among employee organizations or the bargaining relationships between the employer and an employee organization.

(b) The use of such artificial intelligence systems shall not result in the: (i) discharge, displacement or loss of position, including partial displacement such as a reduction in the hours of non-overtime work, wages, or employment benefits, or result in the impairment of existing collective bargaining agreements; or (ii) transfer of existing duties and functions currently performed by employees of the state or any agency or public authority thereof to and artificial intelligence system or automated decision-making tool.

(c) The use of such artificial intelligence system and automated decision-making tool shall not alter the rights or benefits, and privileges, including but not limited to terms and conditions of employment and civil service status. The collective bargaining unit membership status of all existing employees of the board of education shall be preserved and protected.

Prohibits AI systems from impacting employee rights, collective bargaining, employment terms, or transferring duties to AI.

§ 7. Section 3013 of the education law is amended by adding a new subdivision 4 to read as follows: 4.(a) The use of artificial intelligence systems as defined by section one hundred three-e of the state technology law and automated decision- making tools as defined by section five hundred one of the state technology law shall not affect (i) the existing rights of employees pursuant to an existing collective bargaining agreement, or (ii) the existing representational relationships among employee organizations or the bargaining relationships between the employer and an employee organization.

(b) The use of such artificial intelligence systems and automated decision-making tools shall not result in the: (i) discharge, displacement or loss of position, including partial displacement such as a reduction in the hours of non-overtime work, wages, or employment bene- fits, or result in the impairment of existing collective bargaining agreements; or (ii) transfer of existing duties and functions currently performed by employees of the state or any agency or public authority thereof to an artificial intelligence system or automated decision-making tool.

(c) The use of such artificial intelligence system and automated decision-making tool shall not alter the rights or benefits, and privileges, including but not limited to terms and conditions of employment and civil service status. The collective bargaining unit membership status of all existing employees of the board of education, board of cooperative educational services, or board of trustees thereof shall be preserved and protected.

Prohibits AI use from affecting employee rights, employment conditions, or collective bargaining at New York universities.

§ 8. The education law is amended by adding a new section 115 to read as follows: § 115. Use of artificial intelligence systems and automated decisionmaking tools by the state university of New York, the city university of New York, or community colleges. 1. The use of artificial intelligence systems as defined by section one hundred three-e of the state technology law and automated decision-making tools as defined by section five hundred one of the state technology law shall not affect (a) the existing rights of employees pursuant to an existing collective bargaining agreement, or (b) the existing representational relationships among employee organizations or the bargaining relationships between the employer and an employee organization.

2. The use of such artificial intelligence systems and automated decision-making tools shall not result in the: (a) discharge, displacement or loss of position, including partial displacement such as a reduction in the hours of non-overtime work, wages, or employment benefits, or result in the impairment of existing collective bargaining agreements; or (b) transfer of existing duties and functions currently performed by employees of the state or any agency or public authority thereof to an artificial intelligence system or automated decision-making tool.

3. The use of such artificial intelligence system or automated decision-making tool shall not alter the rights or benefits, and privileges, including but not limited to terms and conditions of employment, civil service status, and collective bargaining unit membership status of all existing employees of the state university of New York, the city university of New York, or community colleges shall be preserved and protected.

Prohibits AI and automated tools from impacting employees' rights, positions, or union relationships in civil services.

§ 9. Subdivision 10 of section 80 of the civil service law, as added by chapter 96 of the laws of 2025, is amended to read as follows: 10. (a) The use of artificial intelligence systems as defined by section one hundred three-e of the state technology law and automated decision-making tools as defined by section five hundred one of the state technology law shall not affect (i) the existing rights of employees pursuant to an existing collective bargaining agreement, or (ii) the existing representational relationships among employee organizations or the bargaining relationships between the employer and an employee organization.

(b) The use of such artificial intelligence systems and automated decision-making tools shall not result in the: (i) discharge, displacement or loss of position, including partial displacement such as a reduction in the hours of non-overtime work, wages, or employment benefits, or result in the impairment of existing collective bargaining agreements; or (ii) transfer of existing duties and functions currently performed by employees of the state or any agency or public authority thereof to an artificial intelligence system or automated decision-making tool.

(c) The use of [an] such artificial intelligence system and automated decision-making tool shall not alter the rights or benefits, and privileges, including but not limited to terms and conditions of employment, civil service status, and collective bargaining unit membership status of all existing employees of the state or any agency or public authority thereof shall be preserved and protected.

Specifies effective dates and conditions for sections of the act and their relation to chapter 96 of 2025.

§ 10. This act shall take effect immediately; provided, however, that if chapter 96 of the laws of 2025 shall not have taken effect on or before such date then section nine of this act shall take effect on the same date and in the same manner as such chapter of the laws of 2025, takes effect; provided further, however, that section one of this act shall take effect one year after it shall have become a law; provided further, however, that the amendments to subdivision 10 of section 80 of the civil service law made by section nine of this act shall not affect the expiration and repeal of such subdivision and shall be deemed repealed therewith.