AI Sherpa Talk to us →

Insights · Notable AI Incidents

Notable AI Incidents

Short accounts of AI incidents that matter to organisations in Singapore and APAC: what went wrong, why it matters, and which controls would have helped. Each account is drawn from the regulator's published decision and press reports, linked at the end.

An AI-written email script exposes 95,364 customers' email addresses

Bee Cheng Hiang Marketing · Singapore · Incident 25 April 2026 · PDPC decision published 21 September 2026

Shadow AI AI-assisted coding Personal data
Read the full accountClose the full account

What happened

On 25 April 2026, Bee Cheng Hiang, the Singapore food brand known for its bak kwa, sent a marketing email to its members in batches of 1,000. Every address in a batch sat in the same “To” field, so each recipient could see up to 999 other members' addresses. In all, 95,364 members were affected. Email addresses were the only personal data involved, and there was no evidence of further misuse.

The email went out through a Python script that an employee had written with a generative AI tool, which the Personal Data Protection Commission (PDPC) describes as the employee's personal use of the tool. The prompt asked for a programme to send a mass email from a local list in batches. It did not say that recipients must not see each other's addresses. The script the tool produced was one misplaced bracket away from the correct version: it built one message for the whole batch instead of one per recipient. The employee tested it by checking activity logs and never looked at the contents of a test email.

The difference, as shown in the PDPC decision
As generated: one email to the whole batch
"personalizations": [{"to": [{"email": str(e)} for e in chunk]}]
Corrected: one email per recipient
"personalizations": [{"to": [{"email": str(e)}] } for e in chunk]

The company notified the PDPC two days later, on 27 April. It stopped the mailing, fixed the script, told every affected member, and now has at least two staff check each bulk email before it goes out. This was the first time it had used an AI tool in its business operations. The PDPC said the cause was human error, not a fault in the AI tool, and told The Straits Times this was the first AI-related data breach reported to it. It accepted a voluntary undertaking from the company on 2 September 2026.

Why it matters

  • No AI programme needed. One employee using a general-purpose AI tool for an everyday task was enough. Organisations that have not “adopted AI” still carry this risk.
  • The organisation stays accountable. The PDPC found no fault in the tool and treated the breach as the company's own data protection lapse.
  • Three ordinary gaps. The PDPC found testing that did not check the real output, a single employee with no supervisory review, and no governance framework or policy on staff use of generative AI.
  • A small error, a wide reach. A single bracket exposed 95,364 people. AI-generated code can look right and still behave differently from what was meant.

Controls that would have helped

  • A short policy on staff use of generative AI: which tools are allowed, and which work needs a second pair of eyes.
  • Independent review of any AI-generated code that handles personal data, before it runs.
  • Tests that check what recipients will actually receive, by sending to dummy accounts and opening the email.
  • A technical block in the mail platform on messages that carry many visible recipients.
  • Two-person sign-off for bulk sends.
  • A data protection impact assessment before bringing AI tools into a business process, as the PDPC advises.

What the company has undertaken to do

Under the undertaking, the company is putting in place a governance framework for AI-assisted coding, with independent technical review of AI-generated code that involves personal data, drawing on the 11 AI governance principles in the AI Verify testing framework. It is also adopting a secure software development lifecycle with test emails to dummy accounts, writing down a data breach response procedure, adding automated controls that stop bulk emails with several recipients in one message, and training the technical staff who build and run systems that handle personal data. The PDPC will verify that it complies.

The PDPC's recommendations, explained

The PDPC asks organisations to have three things in place before staff use AI tools: a data protection impact assessment, policies and processes, and testing and review. The undertaking adds a breach procedure and training, and refers to AI Verify. Here is what each involves.

Data protection impact assessment (DPIA)
The PDPC's guide sets out six phases: decide whether a DPIA is needed; plan it; map the personal data involved and where it flows; identify and assess the risks; agree an action plan; then carry it out and check the results. For a new AI tool, that means asking what customer data it will touch before anyone uses it.
AI Verify and its 11 governance principles

AI Verify is a voluntary testing framework, first launched by IMDA and the PDPC in 2022 and now run by the AI Verify Foundation. Organisations use it to check and document their own AI practices against a list of process checks. It is not a certification. Since May 2025 it covers generative AI as well as traditional AI. Its 11 principles are:

  1. Transparency
  2. Explainability
  3. Reproducibility
  4. Safety
  5. Security
  6. Robustness
  7. Fairness
  8. Data governance
  9. Accountability
  10. Human agency and oversight
  11. Inclusive growth, societal and environmental well-being

For AI-written code, accountability, security, robustness, data governance and human oversight matter most: someone owns the code, a second person reviews it, and it is tested before it touches customer data.

Policies, testing and review
A written rule on which AI tools staff may use and for what. Independent review of AI-generated code that handles personal data. Test runs against dummy accounts, checking what actually arrives, before a real send.
A data breach procedure
The PDPC's guide on managing data breaches has four steps: contain, assess, report, evaluate. A breach must be notified if it is likely to cause significant harm or affects 500 or more people. An organisation has up to 30 calendar days to assess a suspected breach, and must tell the PDPC within 3 calendar days of deciding it is notifiable.
Training
Staff who build, review or deploy systems that handle personal data learn these risks and the controls above.

Sources