# The $12k AI Disaster: A Google ADK Postmortem

By Doreen Achterberg · 2026-09-16 · Source: https://www.activepieces.com/blog/the-12k-ai-disaster-a-google-adk-postmortem

---
<aside class="tldr"><p class="tldr-label">Summary</p><p>Google ADK agents cause catastrophic financial and operational damage when autonomous recursive loops trigger massive API overages and corrupt production databases without human oversight or middleware governance.</p><ul><li>Runaway agent loops generated $1,200 in API overages within ninety minutes.</li><li>Recursive errors created 40,000 ghost leads and 14,000 corrupted CRM records.</li><li>Manual data recovery from the integration failure requires 40 man-hours.</li></ul></aside>

## 09:14 AM: the recursive loop that crashed the CRM

### What Google ADK agent access actually entails

When you connect a Google ADK agent to your production environments, you grant the model **autonomous authorization to execute functions** across your software stack through high-level natural language instructions.

Unlike traditional integrations that require explicit field mapping, this access allows the agent to interpret intent and select which API endpoints to hit based on its own internal reasoning.

By granting these permissions without a governance layer to act as a logic gate, you effectively remove the human-in-the-loop requirement for data modification.

This shift from deterministic code to probabilistic execution means the agent can initiate writes to a database that you never explicitly scripted. The tool becomes an unmonitored administrative user with global write access.

An SDK gives an agent code to run, but it does not provide a tenant, a role, or an audit log.

Activepieces addresses this by placing every agent inside the same access model as your employees, where enterprise RBAC and SCIM govern what it may connect to.

Open the run detail view for any agent step to see that each tool call is listed separately with its specific input and output, rather than being collapsed into one opaque result.

### The specific trigger: a feedback loop between Gmail and Salesforce

The failure began when the agent attempted to synchronize communication between the Gmail email service and the Salesforce customer relationship management platform.

The agent monitored the CRM for new leads and sent personalized welcome emails via the mail server, but it lacked a suppression state to recognize its own activity.

The agent reads a 'New Lead' from the CRM, which triggers a 'Data Enrichment' script. This script inadvertently creates a 'Lead Update' event, which the agent then misinterprets as a fresh signal to restart the cycle.

![A lead nurturing workflow with six steps including scheduling, code generation, Google Sheets queries, AI, and delay…](https://ap-marketing-media.fra1.cdn.digitaloceanspaces.com/uploads/bb51a79b-7350-4f9b-bb1b-2ef4adaf0506/the-12k-ai-disaster-a-google-adk-postmortem-scre-d86bf76a.webp)

Because the agent viewed every metadata change as a new prompt for action, it began generating thousands of duplicate entries and outbound emails.

This behavior transformed a simple synchronization task into a **self-sustaining cycle of data inflation** that saturated the API rate limits of both platforms.

### Why the agent didn't stop itself

Redundant tasks continued to execute because large language models lack an inherent understanding of state or operational costs. To the model, every new event looked like a successful fulfillment of its original instructions rather than a redundant error.

![A single large rubber stamp repeatedly hitting the same spot on a piece of paper until the ink has soaked through and the…](https://ap-marketing-media.fra1.cdn.digitaloceanspaces.com/uploads/5b138b6f-f1d2-4b66-9b1c-28cbc0897369/the-12k-ai-disaster-a-google-adk-postmortem-illu-834bc34b.webp)

Without a middleware layer to enforce idempotency, the agent had no mechanism to detect that it was repeating itself. Idempotency is a property where an operation can be repeated multiple times without changing the result beyond the initial application.

Consequently, the system viewed the rising volume of API calls as high productivity rather than a systemic failure. The loop continued until the target platforms enforced a hard lockout to prevent further infrastructure degradation.

## 10:30 AM: Counting the cost of 40,000 ghost leads

The financial and operational fallout of an unmanaged Google ADK agent manifests as a rapid erosion of your cloud budget and the immediate paralysis of your sales department.

When an autonomous agent enters a recursive loop without a governance layer to throttle requests, a high-velocity liability instantly replaces the convenience of automation.

### The $1,200 API overage in ninety minutes

Unmonitored agent loops generate infrastructure costs that far outpace the predictable subscription fees of governed middleware.

<blockquote class="pull"><p>When an autonomous agent enters a recursive loop without a governance layer to throttle requests, a high-velocity liability instantly replaces the convenience of automation.</p></blockquote>

[Platform](https://platform.claude.com/docs/en/api/beta/organization/cost_report/retrieve)'s analysis puts the cost of a managed platform like the Activepieces Team tier at $200 USD per month, providing a fixed ceiling for operational spend.

![Monthly cost of mid-tier workflow automation](https://ap-marketing-media.fra1.cdn.digitaloceanspaces.com/uploads/6caf2fc2-a028-492d-87fc-0b16920abdaf/the-12k-ai-disaster-a-google-adk-postmortem-pict-7ccc74ab.svg "Source: NodeSparks")

### Comparing middleware costs to direct access

In just ninety minutes, the ADK agent in this scenario incurred **$1,200 in API overages**. You spent six months of your automation budget in less than two hours.

This disparity highlights the risk of "pay-as-you-go" models without circuit breakers. A [Zapier Pro](https://www.nodesparks.com/data/automation-platform-pricing) account at $103 USD per month or a [Make Pro](https://www.nodesparks.com/data/automation-platform-pricing) plan at $53 USD per month acts as a financial buffer.

Every agent tool call and the data it acted on is traced step by step in Activepieces, alongside the deterministic flow steps running next to it.

This trace exports as audit logs and event streams into the SIEM your security team already runs, ensuring an agent's decisions are reviewed the same way a workflow's steps are.

Companies like MoneyGram and FundingSocieties run this in production to maintain this level of observability across their automated processes.

### Budget alerts and rate-limiting

The direct ADK connection offers no such protection. This leaves you fully exposed to runaway consumption costs.

NodeSparks notes that even lower-cost options such as n8n Starter at $21 USD per month or Pipedream Professional at $19 USD per month include rate-limiting features.

### Visualizing the record explosion

Before the cost spiked from $0 to $12,000, these would have alerted you, effectively capping the damage before it reached catastrophic levels.

The following chart illustrates the compounding nature of these costs as the record count exploded between 09:15 and 10:45.

### The cost of redundant retries

The vertical climb in expenditure demonstrates that without a middleware layer to intercept redundant calls, the agent treats every error as a mandate to retry, regardless of the billing implications.

### Data integrity: Cleaning 14,000 corrupted records

Recursive agent errors don't just create noise. They overwrite valid customer data with "ghost" attributes that render the database untrustworthy.

In this instance, the agent generated **14,000 corrupted records** within the CRM, which means your marketing team must now pause all outbound campaigns to avoid emailing non-existent or mislabeled leads.

Because the ADK lacked a transformation schema, it mapped raw JSON strings directly into the "Last Name" and "Email" fields of the Salesforce CRM. This forced a manual data recovery process that typically takes 40 man-hours to complete.

### The opportunity cost of a locked sales pipeline

When the API provider detected the surge of **40,000 ghost leads**, it revoked your access tokens. A direct-to-app agent failure results in a hard lockout from critical services, stopping all revenue-generating activity.

![A workflow automation builder showing a Lead Nurturing flow with six connected steps and trigger settings panel on the right.](https://ap-marketing-media.fra1.cdn.digitaloceanspaces.com/uploads/fa4624a2-4195-47b7-b35b-4370f9ccce6d/marketing-to-sales-handoff-2026-automation-guide-d020bc71.webp)

For the remainder of the business day, your 12-person sales team was unable to log into their primary prospecting tools. This total service suspension represents a complete loss of sales velocity, transforming a technical integration error into a measurable deficit in the quarterly pipeline.

## 12:00 PM: Why standard OAuth scopes failed to prevent the crash

Standard OAuth scopes fail to prevent system crashes. They authorize the identity of the agent rather than the intent of its specific actions.

While a human user understands the context of a "write" command, a Google ADK agent interprets a broad permission as a mandate to execute every available API call until it completes a task.

### The 'all-or-nothing' nature of Google ADK permissions

To ensure functionality, Google ADK integrations typically rely on **coarse-grained OAuth scopes** that grant an agent full read-write access to a workspace.

This lack of granular control means that an agent with access to a CRM inherits the ability to modify every record the authenticated user can touch.

This effectively removes the "least privilege" safety net. The following table illustrates how these traditional permissions fail when processed by an autonomous agent:

| CRM Permission | Standard OAuth Risk | Agentic Failure Mode |
| :--- | :--- | :--- |
| Write Access | Unauthorized edits by a compromised user | Recursive loop creating 40k records due to a logic error |
| Delete Access | Accidental single-record deletion | Mass purging of "duplicate" leads that were actually distinct marketing segments |
| Bulk API Access | Data exfiltration by a malicious actor | Rapid exhaustion of daily API credits through redundant polling |

This gap between permission and execution creates a structural vulnerability where the agent operates with the power of an admin but the judgment of a script.

### Why agents struggle with destructive action confirmation

Agents lack the native ability to distinguish between a constructive update and a destructive recursive loop. They prioritize the mathematical completion of a prompt over system stability.

<blockquote class="pull"><p>Agents lack the native ability to distinguish between a constructive update and a destructive recursive loop.</p></blockquote>

A "delete" command would trigger a human-in-the-loop validation in a standard middleware environment. A direct ADK connection treats the absence of a "stop" signal as an implicit "continue."

This results in a feedback loop where the agent attempts to fix a perceived error by generating more data. This in turn triggers more API calls, eventually leading to the service suspension described previously.

### Daily API limits for Google AI users

To prevent infrastructure abuse, Google enforces strict rate limits on its AI services. An unmonitored agent can burn through a monthly operational budget in minutes.

When an agent enters a recursive state, it hits these ceilings rapidly. This causes immediate service throttling so legitimate users are locked out of the tool. It also leads to increased overage charges and authentication token revocation.

## 02:00 PM: How teams typically secure AI agent integrations

Securing AI agent integrations requires an architectural decoupling that prevents the Google ADK from interacting directly with the application’s primary database or administrative API endpoints.

Without this separation, an agent operates with the full permissions of its API key. A single logic error in a prompt can trigger an irreversible cascade of data modifications across the entire production environment.

### Implementing human-in-the-loop (HITL) for high-stakes writes

Human-in-the-loop (HITL) protocols are a physical break in the automation chain, requiring a verified user to sign off on any action that alters or deletes data.

By forcing the agent to generate a "pending request" rather than executing a command, you ensure that a senior developer or manager reviews the output.

This prevents the agent from hallucinating a destructive SQL query or mass-deleting customer records during a recursive loop. This transition from autonomous execution to supervised suggestion transforms the agent from a potential liability into a productivity tool that still requires human accountability.

### Using middleware as a protocol buffer

Sitting between the Google ADK and the target application, a middleware layer acts as a governance vault to intercept and validate every outgoing call.

This structure prevents the "black box" effect by logging every request in a centralized audit trail, which allows your security team to identify exactly when an agent begins to deviate from its intended behavior.

### Intercepting agent calls through function definitions

The physical decoupling is achieved by registering the middleware endpoint as the exclusive tool or function within the agent configuration. Instead of giving the model direct access to external application programming interfaces, developers define a schema that routes all operational requests to the proxy URL.

When the model determines an action is necessary, it generates a structured payload intended for the middleware rather than executing a direct write to the production database.

This mechanism ensures that every autonomous decision is intercepted, parsed, and validated before any external system receives the command.

To contain these risks, you should follow a three-step containment strategy. You proxy the API key through a middleware vault so the agent never possesses the actual credentials for the production environment.

![A heavy, locked vault door with a small, narrow mail slot in the center; a hand on one side is sliding a plain envelope…](https://ap-marketing-media.fra1.cdn.digitaloceanspaces.com/uploads/a430dbfc-c143-4670-9c48-3ba622852cf3/the-12k-ai-disaster-a-google-adk-postmortem-illu-cf9e73a9.webp)

### Containment strategies for autonomous agents

* Set hard rate limits at the integration layer.
* Insert a 'Human-in-the-loop' approval step for any write operation.

This sequence ensures that even if the agent’s logic fails, the blast radius is confined by the pre-set limits of the middleware.

### Monthly cost of mid-tier workflow automation

The financial commitment for securing these integrations varies based on the volume of operations and the level of network isolation required. The basic tier limits the number of active workflows, which forces you to prioritize only the most critical automations.

Developers are alerted to failures before they impact the end-user experience because the professional tier includes advanced error handling. The enterprise tier offers dedicated IP addresses and SAML authentication.

## 04:00 PM: How Activepieces governs Google ADK agent actions

Activepieces acts as a programmable firewall between Google ADK agents and your core data by enforcing manual approval steps and rate limits that the agent can't override.

Without this intermediary, an agent operates with the full permissions of its API key, turning a simple logic error into an unmonitored write-loop across your production environment.

### Building a circuit breaker for recursive loops

Activepieces prevents runaway API costs by allowing administrators to hard-cap the number of times an agent can trigger a specific workflow within a set window. This is critical because the daily quotas for Google AI services vary significantly by tier.

* Google AI Pro provides 1,500 requests per day.
* Code Assist Standard also caps at 1,500 requests.
* Gemini Code Assist offers a lower ceiling of 1,000 requests.
* Gemini API Free is limited to 250 requests.

![Daily API limits for Google AI users](https://ap-marketing-media.fra1.cdn.digitaloceanspaces.com/uploads/45aca2b9-0bab-4e7f-860e-698e3c9b7459/the-12k-ai-disaster-a-google-adk-postmortem-stac-6c8d9e8f.svg "Source: GeminiCLI")

### Granular 'Read-Only' vs 'Write' zones

Governance requires separating the agent’s ability to "see" data from its ability to "change" it through defined connection scopes. In Activepieces, you can route an ADK agent through a "Read-Only" flow when querying the Salesforce CRM platform.

This ensures it can extract lead data without the risk of overwriting historical contact records. When an agent needs to perform a "Write" action, such as updating a ticket in the Jira project management tool, you can insert a "Wait for Approval" step.

![A two-step workflow connecting Google Sheets to MailChimp, with the first step's configuration panel open on the right.](https://ap-marketing-media.fra1.cdn.digitaloceanspaces.com/uploads/1039a238-65f0-4dd0-9ffd-f32a063be786/chatgpt-apps-sdk-how-to-build-business-workflows-15fe75ec.webp)

Before the agent’s payload is delivered to the production API, an auditor must click "Approve" to satisfy the human-in-the-loop requirement.

### Auditing the 'Black Box' with execution logs

Activepieces replaces the opaque nature of direct ADK calls with a versioned history of every logic change and execution path.

The flow builder allows you to track exactly how an agent’s instructions have evolved over time, which is essential for identifying which version of a prompt began causing data inconsistencies.

The Version History panel is an immutable record of these changes, showing which iterations were active during specific failure windows.

In the screenshot, Version #2 (Today, 11:59 AM) is flagged with a yellow status indicator. This signifies a draft or a "caution" state that hasn't yet replaced the stable Version #1 (Today, 11:10 AM).

![Flow History panel showing two versions of a flow with timestamps and status indicators](https://ap-marketing-media.fra1.cdn.digitaloceanspaces.com/uploads/17dfdf51-685f-4316-aaee-1dd5f16dc705/what-is-a-webhook-payload-structure-and-examples-f2789ff4.webp)

By maintaining this trail, an auditor can see that the "Code" step was modified between 11:10 AM and 11:59 AM.

This provides a clear starting point for a forensic review. This visibility ensures that when an agent's output changes, you can distinguish between an update to the model's weights and a manual change to the workflow logic.

## Related reading

- [A Postmortem on Data Leaks and Secure Automation](https://www.activepieces.com/blog/a-postmortem-on-data-leaks-and-secure-automation)

## References

- [GeminiCLI](https://geminicli.com/docs/resources/quota-and-pricing/)
- [NodeSparks](https://www.nodesparks.com/data/automation-platform-pricing)
