Blog Post

Human Oversight Still Matters in AI

Loading…

Nancy DiGioacchino

VP, Quality Management and Global Compliance

In April 2026, FDA issued a Warning Letter to Purolea Cosmetics Lab that appears to mark an important first in AI-related enforcement: the first time FDA cited the inappropriate use of AI within a regulated process as a compliance failure.

What FDA Found

Purolea used AI agents to help develop drug product specifications, procedures, and master production and control records.

The problem was not the use of AI. It was that AI-generated materials were not adequately reviewed for accuracy and cGMP compliance before being used.

FDA also found that when required process validation had not been performed, the company explained that its AI agent had not identified the requirement.

The takeaway is straightforward: AI can support regulated work, but it does not replace human accountability.

A Similar Theme in the EU AI Act

The same principle is reflected in the EU AI Act.

Article 14 requires human oversight for high-risk AI systems. Article 50, which applies transparency requirements to certain AI systems beyond the high-risk category, also recognizes the role of human review and editorial control for certain AI-generated content.

The exact requirements depend on how AI is being used, but the broader regulatory direction is consistent: organizations remain responsible for the outputs and decisions supported by AI.

Florence’s Approach to AI

At Florence, responsible and compliant use of AI is built into how we design AI capabilities.

Our approach is grounded in a few core principles:

  • Human oversight remains central. AI supports decisions and workflows but does not replace clinical, regulatory, or professional judgment. A person remains responsible for reviewing outputs before they move regulated work forward.
  • Customer data remains protected. AI capabilities operate within Florence’s secure environment and existing permission controls.
  • Customers remain in control. AI capabilities that interact with customer data are enabled through explicit customer opt-in.
  • AI is designed for regulated clinical research. Florence evaluates AI capabilities against the regulatory and compliance frameworks that govern our industry, including GxP requirements, 21 CFR Part 11, HIPAA, GDPR, and the EU AI Act.

FDA’s Purolea Warning Letter reinforces a principle that is already central to Florence’s approach: AI can accelerate regulated work, but appropriate controls, human review, and accountability must remain in place.

Want to Learn More About AI at Florence?

Learn more about how Florence is bringing AI into clinical research while keeping human oversight, data protection, and customer control at the center.

Learn about AI at Florence

You May Also Like

Video

Tales of Trial Management: Andrea Bastek

Watch Now →

computer screen visual

Guide

5 Questions to Ask Before Selecting an eTMF

Read Now →

Blog

What Successful AI Adoption in Clinical Research Actually Looks Like

Read Now →