← Back to perspectives
/Intellicon FDE Team

Why Can an AI Agent Still Leak Customer Data During Isolated Testing?

Development credentials mistakenly placed in production can expose customer data to testing risks. First verify connection destinations and accessible data, then test whether unauthorized actions are blocked, and assign someone to review the logs.

AIEgentWrX導入
Why Can an AI Agent Still Leak Customer Data During Isolated Testing?

When preparing to let an AI Agent help sales staff look up quotes, a team may first set up a test environment and connect it to a database to check whether the workflow runs properly. But if the test account can still read production customer data, or the test environment can still connect to unapproved services, managers should not approve testing simply because they have been told that the environment is “isolated.” Intellicon Solutions recommends first verifying which services are reachable and which data is accessible, then deciding what work the AI Agent should perform.

Why Was Customer Data Downloaded During Testing in an Isolated Environment?

According to iThome, an AI Agent inadvertently left an isolated test environment while researchers were running security benchmarks through ARTEX, resulting in a data breach at WorkOS. After comparing technical evidence and internal records, WorkOS and a third-party forensic team confirmed the researchers’ account. The compromised internal business application was still running on an older version of the platform, with overly permissive access controls. Its application programming interface (API, an entry point through which software calls a service) was directly accessible over the public internet. In addition, development credentials (information software uses to prove its identity) had mistakenly been placed in the production environment and could still pass authentication (the process of verifying a user’s or program’s identity).

The report also noted that the API lacked rate limiting, which restricts the number of requests allowed within a given period. Once authenticated, the AI Agent was therefore able to extract large amounts of data in a short time. WorkOS did not discover the incident until it detected unusual system usage on October 30, by which point customer data had already been downloaded. However, the report did not explain how the AI Agent left the isolated environment or provide details of the isolation configuration. The incident therefore cannot be blamed entirely on the AI Agent, nor does it establish that any particular isolation method is necessarily ineffective.

Enterprises preparing to adopt an AI Agent should check the test environment and production systems separately: does the test environment actually restrict connections, and do production systems reject credentials and data requests they should not accept? The project owner should ask the administrators responsible for each side to provide their own verification results. Confirming that the test environment has been set up is not a reason to skip checks on production systems.

Before Testing, List What the AI Agent Can Connect to and Read

IT, security staff, and business owners should jointly confirm the scope of testing and document it in a checklist for subsequent verification. In a quotation workflow, for example, sales staff should first specify which items they need to look up and which pricing fields they need to use. IT can then configure the permitted services and accessible data, rather than opening up the entire customer dataset for convenience.

  • Permitted services and network paths: List the services, domains, and network devices along the connection paths required for testing. If the task does not require external queries, do not allow other outbound connections simply for convenience.
  • Credentials used for testing: Credentials are the information software uses to prove its identity. Create dedicated test credentials and record who is responsible for them and when they will be disabled, rather than reusing production accounts.
  • Data permitted for reading: List the fields and data scope that may be queried. First use test data containing no real customer information to verify that the workflow is viable. Any subsequent expansion of the data scope should require prior approval from the data owner.

Once the checklist is confirmed, test whether actions that should be denied are actually blocked, staying within the scope authorized by the enterprise. This includes checking whether unauthorized connections are blocked and whether the test account is genuinely unable to read out-of-scope data. Testers should also retain the results: successfully completing a quote lookup only proves that the workflow works, not that the system will block other unauthorized connection and read requests.

In EgentWrX, for example, the outbound allowlist specifies the domains that may be reached through outbound connections. The platform denies all outbound connections by default, then allows approved domains individually. Internal company database integrations are read-only and cannot write data. Even with read-only access, however, an AI Agent may still obtain sensitive content. Administrators should therefore allow only the domains required for the task and ask the database owner to restrict the scope of readable data. Read-only access should not be treated as protection against data leaks.

Who Checks Production Systems, and Who Reviews Test Logs?

Production system owners should separately inventory externally exposed APIs, check whether older internal applications remain accessible from outside, and remove any development credentials mistakenly placed in production. They also need to review access controls, which determine which identities can perform which actions and read which data, then configure API rate limits and verify that they work. These tasks remain the enterprise’s responsibility; adopting a platform does not mean they have already been handled.

The project owner should also specify in the test plan who will review the logs. Before testing, confirm what records the platform, the APIs of connected services, and network devices each retain, and whether those records can be correlated by test time and account. Run through a query workflow first to confirm that reviewers can use the logs to identify the actual connection destinations and the scope of data accessed before expanding testing. If the records are incomplete, address the gaps first rather than waiting until an incident to look for evidence.

EgentWrX provides exportable audit logs that administrators can use to review platform operations. A hash chain also allows them to verify log integrity by checking whether linked records have been altered. These logs support reviews of platform-side activity, but having logs does not mean anomalies can be detected in real time, nor can platform logs replace records from connected services and network devices. Rehearse the log export and integrity verification procedures first, and confirm that reviewers can correlate records across all sources.

Before approving testing, managers should confirm who will review the logs, what conditions require testing to stop, and who has the authority to approve its resumption. If an unexpected connection occurs or data outside the permitted scope is accessed, the designated person should stop testing and preserve the records. Once IT and security staff have identified the cause and reconfirmed that the restrictions work, the person responsible can decide whether testing should resume.

FAQ

Is an isolated test environment enough to let an AI Agent perform tasks safely?

No. You also need to verify that connection and data access restrictions work. Within the authorized scope, confirm that unapproved services cannot be reached and that the test account cannot read out-of-scope data. Ask production system owners to check whether development credentials can still authenticate. An environment’s label alone does not establish that it is secure.

Is setting a database to read-only enough to prevent data leaks during testing?

No. Read-only access restricts changes to data, but sensitive content can still be read. Start with test data that contains no real customer information, then restrict accessible fields and data scope according to the task’s needs. Obtain the data owner’s approval before expanding that scope.

Which logs should be reviewed during testing?

Review platform audit logs, API access logs from connected services, and network connection logs. Because each source records different information, assign someone before testing to confirm whether the records can be correlated by time and account. If unexpected access is detected, stop testing and preserve the logs before investigating the cause.

References

30 minutes to map out which work to hand to AI first

Want every employee to have their own AI teammate?

A consultant will be in touch shortly.

ISO/IEC 27001:2022 Certified ISMS