New York Department of Financial Services proposes insurance circular letter regarding use of artificial intelligence systems and external consumer data and information sources in insurance underwriting and pricing

On January 17, 2024, The New York Department of Financial Services (“DFS”) issued a proposed insurance circular letter regarding insurers’ use of external consumer data and information sources (“ECDIS”) and artificial intelligence systems (“AIS”) in underwriting and pricing. In recognizing that the use of ECDIS and AIS in underwriting and pricing could be beneficial to insurers and the public by expediting and increasing the accuracy of underwriting and pricing, the DFS also recognized risks associated with such practices. The DFS noted:

At the same time, ECDIS may reflect systemic biases and its use can reinforce and exacerbate inequality. This raises significant concerns about the potential for unfair adverse effects or discriminatory decision-making. ECDIS may also have variable accuracy and reliability and may come from entities that are not subject to regulatory oversight and consumer protections. Furthermore, the self-learning behavior of AIS increases the risks of inaccurate, arbitrary, capricious, or unfairly discriminatory outcomes that may disproportionately affect vulnerable communities and individuals or otherwise undermine the insurance marketplace in New York.

Based on this, the DFS articulated its expectations that insurers would develop and manage their use of ECDIS, artificial intelligence systems, and other predictive models in underwriting and pricing insurance policies and annuity contracts.

The DFS has proposed guidelines for insurers to meet the expectations of the department. 

An insurer should not use ECDIS or AIS for underwriting or pricing purposes unless the insurer can establish that the data source or model, as applicable, does not use and is not based in any way on any class protected pursuant to Insurance Law Article 26. Moreover, an insurer should not use ECDIS or AIS for underwriting or pricing purposes if such use would result in or permit any unfair discrimination or otherwise violate the Insurance Law or any regulations promulgated thereunder. Included among these obligations, the DFS outlined the following:

1. Data and Actuarial Validity: insurers should be able to demonstrate that the ECDIS are supported by generally accepted actuarial standards of practice and are based on actual or reasonably anticipated experience, including, but not limited to, statistical studies, predictive modeling, and risk assessments.
2. Unfair and Unlawful Discrimination: insurer should not use ECDIS or AIS in underwriting or pricing unless the insurer has determined that the ECDIS or AIS does not collect or use criteria that would constitute unfair or unlawful discrimination or an unfair trade practice under State or Federal law. A comprehensive assessment of whether an underwriting or pricing guideline derived from ECDIS or AIS unfairly discriminates between similarly situated individuals or unlawfully discriminates against a protected class should, at a minimum, include the following steps:

1. 1. Assessing whether the use of ECDIS or AIS produces disproportionate adverse effects in underwriting and/or pricing on similarly situated insureds, or insureds of a protected class. If there is no prima facie showing of a disproportionate adverse effect, then the insurer may conclude its evaluation.
   2. If there is prima facie showing of such a disproportionate adverse effect, further assessing whether there is a legitimate, lawful, and fair explanation or rationale for the differential effect on similarly situated insureds. If no legitimate, lawful, and fair explanation or rationale can account for the differential effect on similarly situated insureds, the insurer should modify its use of such ECDIS or AIS and evaluate the modified use of ECDIS or AIS.
   3. If a legitimate, lawful, and fair explanation or rationale can account for the differential effect, further conducting and appropriately documenting a search and analysis for a less discriminatory alternative variable(s) or methodology that would reasonably meet the insurer’s legitimate business needs. If a less discriminatory alternative exists, the insurer should modify its use of ECDIS or AIS.

In analyzing potentially unfair or unlawful discriminatory practices, insurers should appropriately document the processes and reasoning behind their testing methodologies and analysis for unfair or unlawful discrimination associated with the use of ECDIS and AIS and the complexity and materiality of such ECDIS and AIS. Insurers should be prepared to make such documentation available to the DFS.

Prior to putting AIS into production, should administer testing and analyze for unfair or unlawful discrimination. Such testing and analysis should be done whenever materials updates or changes are made to either the ECDIS or AIS. In addition to testing, qualitative and quantitative assessments should be conducted by insurers.

In addition to the testing and analysis components, insurers will also be subject to corporate governance and risk management requirements. These requirements include Board and management oversight of the insurers use and implementation of ECDIS and AIS. Insurers that use ECDIS or AIS should formalize their development and management of ECDIS and AIS in written policies and procedures consistent with the DFS’s Circular Letter.

Insurers should recall that they retain responsibility for understanding any tools, EDCIS, or AIS used in underwriting and pricing for insurance that were developed or deployed by third-party vendors to ensure that such tools, EDCIS, or AIS comply with all applicable laws, rules, and regulations. To ensure appropriate oversight of third-party vendors, DFS has stated that insurers should develop written standards, policies, procedures, and protocols for the acquisition, use of, or reliance on ECDIS and AIS developed or deployed by a third- party vendor. Additionally, insurers should put in place procedures for reporting any incorrect information to third-party vendors for further investigation and update, as necessary. Insurers should also develop procedures to remediate and eliminate incorrect information from their AIS that the insurer has identified or has been reported to a third-party.

Finally, the DFS Circular Letter makes clear that transparency in communicating with insureds or potential insureds is required. Where an insurer is using ECDIS or AIS, the reason or reasons provided to the insured or potential insured, or a medical professional designee, should include details about all information upon which the insurer based any declination, limitation, rate differential, or other adverse underwriting decision, including the specific source of the information upon which the insurer based its adverse underwriting or pricing decision.

The DFS is requesting comment on the Circular Letter. Insurers wishing to submit comments or feedback may do so on or before March 17, 2024.