← All articles

Security

Rescue Playbook for Inherited AI-Generated Codebases

6 min read

Updated

A practical 7-step guide to audit secrets, auth, data layer, LLM calls, validation, tests, and deployment in AI-built applications.

TL;DR

Inherited an AI-generated codebase? Follow this 7-step cleanup order before adding new features.

  • Step 1 – Secrets: Locate and rotate exposed keys; they can drain money instantly.
  • Step 2 – Auth: Reinstate row-level security, move role checks server-side, verify JWTs.
  • Step 3 – Data Layer: Fix N+1 queries, add indexes, implement pagination, trim column selection.
  • Step 4 – LLM Calls: Add rate limits, spend caps, retries, timeouts, streaming where needed.
  • Step 5 – Validation: Use schemas (e.g., Zod) for all inputs and external responses.
  • Step 6 – Tests: Write integration tests for the most critical user flows.
  • Step 7 – Deployment: Verify env vars, CORS, error tracking, logs, health checks, backups.

Who Benefits

Senior engineers, agency owners, and founders who have inherited (or built) a working-but-fragile AI-generated app.

1. Audit the Secrets

Why it matters: Exposed API keys can generate multi-digit bills within minutes.

Typical findings

  1. .env files committed to Git.
  2. API keys (OpenAI, Anthropic, etc.) hard-coded in frontend files.
  3. Supabase service_role keys in client components.
  4. Stripe secret keys bundled with the client.
  5. AWS access keys in config files.

Remediation steps

  1. Run a secret scanner such as gitleaks detect or trufflehog filesystem ..
  2. Rotate every leaked secret immediately via the provider dashboard.
  3. Store secrets in proper environment variables; split server-side and client-side keys where required.
  4. Add a pre-commit hook to block future secret commits.

2. Audit the Auth

Typical symptoms

  • Row-level security (RLS) disabled on one or more tables.
  • Role checks (if (user.role === 'admin')) performed only on the client.
  • Missing JWT verification on mutating API routes.
  • Auth middleware that bypasses checks.
  • User IDs read from request bodies instead of validated sessions.

Fix order

  1. Re-enable RLS for every table.
  2. Write granular policies (e.g., “users can read/write their own rows”).
  3. Move all role checks to the server.
  4. Add a global auth middleware to enforce verification by default.
  5. Replace body-derived user_id with the ID from the validated session.

Do not add MFA or session rotation yet; focus on the basics first.

3. Audit the Data Layer

Common problems

  • N+1 queries on list views.
  • No indexes beyond primary keys.
  • Missing pagination on list endpoints.
  • SELECT * on wide tables when only a few columns are needed.
  • Full-table scans on every render.

How to address

  1. Enable your database’s slow-query log and sort by total execution time.
  2. Add missing indexes.
  3. Batch N+1 calls (e.g., fetch needed records in a single query).
  4. Implement cursor-based pagination.
  5. Select only the required columns in SQL statements.

4. Audit the LLM Calls

New in 2026: Many AI-generated apps forget to manage LLM usage properly.

Things to check

  • Rate limiting (often absent).
  • Monthly spend caps (often absent).
  • Retry logic with exponential backoff.
  • Reasonable timeouts (default SDK timeout is usually 10 minutes).
  • Streaming for long responses.
  • Prompt-injection guards on user-provided text.

Actions

  • Set a hard monthly spend cap in the provider dashboard.
  • Add per-user rate limits at the application layer.
  • Wrap each LLM call in a retry helper with a ~30-second timeout.
  • Enable streaming for responses that benefit from it.

5. Audit the Validation

AI tools frequently skip runtime validation.

Symptoms

  • Free-text fields where enums belong.
  • Dates stored as strings.
  • IDs inconsistently typed (number vs. string).
  • Unparsed JSON payloads from clients.

Remediation

  • Introduce a schema validator (e.g., Zod) for every API route’s input.
  • Validate external API responses before using them.
  • Align database column types with the defined schemas.

A half-day of work here prevents a year of obscure bugs.

6. Audit the Tests

Typical state: Skeleton test files with trivial assertions.

Strategy

  1. Identify the 10 most critical user flows (those that would cause immediate impact if broken).
  2. Write integration tests covering these flows.
  3. Add a regression test for each bug discovered during the audit.

Avoid chasing high coverage percentages initially; focus on high-risk paths.

7. Audit the Deployment

Checklist

  • Ensure environment variables are correctly configured in the hosting platform (Vercel, Render, Railway, Fly, etc.).
  • Set explicit CORS origins instead of *.
  • Wire up an error-tracking service (Sentry, Highlight, etc.).
  • Route logs to a searchable system.
  • Provide a health-check endpoint that pings the database.
  • Verify regular database backups.
  • Include a proper robots.txt and sensible Cache-Control headers.

Consider adding preview deployments to speed up future changes.

Recommended Toolset for the Audit

  • Claude Code – Structured code-base read pass; generates security and architecture reports.
  • Cursor – Inline diff editor for applying fixes.
  • Semgrep or Snyk – Security scanning (free tiers sufficient for most small apps).
  • Gitleaks or TruffleHog – Secret scanning (run first).
  • Sentry – Observability and error tracking (free tier works for startups).
  • Lovable – Rebuild UI layer while preserving backend logic, if needed.

Typical total cost: under $100 / month for a small team.

DIY vs. Hiring an Agency

Do it yourself if

  • The app is pre-launch or has fewer than 100 users.
  • You have a senior engineer who can dedicate part-time effort for about a month.
  • You can rotate keys and adjust RLS policies yourself.

Hire a rescue agency if

  • Paying customers are affected and downtime costs money.
  • You’re a non-technical founder.
  • Exposed secrets need immediate rotation.
  • You’re conducting due-diligence for an acquisition.

Agency rescues typically cost $4 k – $25 k for a small SaaS, depending on codebase size and urgency.

30-Day Plan Overview

WeekFocus
1Triage: map routes, schema, run secret scan, list external services, create risk register.
2Secrets & Auth: rotate keys, enable RLS, write policies, move role checks server-side, add auth middleware.
3Data Layer: analyze slow queries, add indexes, fix N+1, implement pagination, tighten column selection, set API spend caps.
4Tests & Observability: write integration tests for critical flows, set up Sentry, ensure logs & health checks, update README.

At the end of week 4 you should have a stable, maintainable codebase—exactly what you need before adding new features.

FAQ

What does “AI-generated” or “vibe-coded” mean?
It describes a workflow where developers accept most AI-produced code without thorough review, often resulting in hidden bugs, missing security controls, and performance issues.

What’s the biggest risk in an inherited AI-generated codebase?
Committed secrets. They can be abused instantly, leading to costly bill-ups.

How long does a full rescue take?
For a small SaaS (< 30 k lines) expect four weeks. Larger projects scale roughly linearly.

Should I rewrite the app?
Prefer refactoring. Rewrites discard existing product decisions and user data. Keep the schema, routes, and core business logic; replace only the problematic layers.

Which tools are essential?
Claude Code (read pass), Cursor (diff editing), Semgrep/Snyk (security), Gitleaks/TruffleHog (secrets), Sentry (observability). Total cost stays under $100 / month for modest usage.

When is hiring an agency the right choice?
When you have paying users, lack senior engineering resources, or need immediate secret rotation.