Hitesh Dubey
Head of DevOps and Infrastructure
Every regulated financial firm we work with runs the same security review against every new AI vendor. The wording changes; the questions do not. This checklist is the working version we hand to CIOs and risk teams so their internal review does not have to start from scratch.
Scope of the review
Before the checklist even starts, agree three things: which product component is in scope, which data classes it will touch, and which environments it will run in. Reviews that skip this always end up circular.
Data handling boundary
The first question every reviewer asks: what leaves the boundary?
- Where does customer content sit at rest, and who holds the encryption keys?
- Which components make outbound network calls, and can outbound be disabled without breaking the product?
- What logging does the vendor retain about our queries, and for how long?
- What happens to our content if we terminate — deletion timeline, evidence of destruction?
Model provenance
The LLM is the moving part with the biggest supply-chain question.
- Which model is being called and from where? Is the endpoint under our contract or the vendor’s?
- Is our content ever used to train, evaluate, or fine-tune any model?
- What is the incident path if the model provider suffers a security event?
Access control and audit
The AI product is only as secure as the identity layer sitting in front of it.
- Does the product integrate with our SAML 2.0 / OIDC identity provider?
- Can we scope retrieval to per-user and per-desk access boundaries?
- What audit events are emitted, and can we ingest them into our SIEM in real time?
Third-party risk (SYSC / DORA)
For FCA-, PRA-, and DORA-scoped firms, the vendor sits in the third-party risk register.
- Does the vendor hold SOC 2 Type II and ISO 27001? Is a recent audit report available under NDA?
- What is the operational resilience posture — HA topology, RTO, RPO, DR site?
- What contractual coverage — DPA, security addendum, exit assistance — is standard?
Ongoing operational governance
A security review is not a one-off. Bake ongoing checks into the vendor governance calendar.
- Quarterly review of retrieval scope and access model against staff movement.
- Annual review of subprocessor list and DPA.
- Change-window alignment for vendor releases and out-of-band security patches.