We Scanned 30 Live Startups. Here’s What We Found.

In May 2026, we picked 30 startups at random off Product Hunt and pointed Greywatch's AI agents at them. No inside access - just the public product. About 20 minutes per target. More than 2 in 3 had a serious, exploitable vulnerability already sitting in their live product. That’s 21 confirmed findings - every one manually exploited to proof by a security engineer. 14 rated severe or critical. 7 high.

What did we find?

The most common impact categories across those 21 confirmed findings:

• Account takeover - authentication flaws allowing access to any user account without credentials
• Customer data and PII exposure - endpoints returning other users’ personal data without authorization checks
• Database access - unauthenticated or improperly scoped endpoints returning raw database records

The 9 startups with no confirmed findings were not necessarily secure. They were either well-defended against external probing, or the agents did not surface a finding within the 20-minute window. A longer engagement would have tested more surface area. The 21 confirmed findings were from 20 minutes of external probing, no credentials, no source code access.

What kinds of vulnerabilities?

Broken access control and IDOR (Insecure Direct Object Reference) was the dominant category, representing 40% of all findings. These vulnerabilities occur when an API endpoint returns data based on an ID in the request without verifying that the requesting user is authorized to access that record. The fix is simple in principle - check that the logged-in user owns the resource before returning it - but authorization logic is easy to miss on individual endpoints and even easier to miss when a codebase is growing fast. Scanners catch almost none of these because the vulnerability only reveals itself when you actually probe the relationship between a user token and another user’s data.

Hardcoded secrets and API keys accounted for 30% of findings - the second largest category. Client-side bundles, JavaScript source maps, public GitHub repositories, and misconfigured environment variable handling were the most common sources. An AWS secret key visible in a client-side bundle gives an attacker direct access to your cloud infrastructure. These findings are fast to find, fast to exploit, and catastrophic to remediate after exposure.

Injection vulnerabilities (SQL, command, and template injection combined) came in at 10%. These are the classic vulnerabilities that most teams believe they have covered - and most do, for their main flows. The findings here were typically in less-travelled paths: admin endpoints, import/export features, reporting APIs, and integrations that were built quickly and never revisited.

How did we run it?

The methodology was deliberately constrained to reflect realistic external attacker conditions. 30 targets, pulled from Product Hunt, no inside access, no credentials, no prior knowledge of the tech stack. The agents were given approximately 20 minutes per target - enough time to enumerate the public attack surface, probe the most common vulnerability categories, and surface candidate findings for human review.

Every candidate finding from the agents was reviewed by a security engineer before being classified as confirmed. The engineer manually replicated the exploit - running the actual HTTP request, observing the actual response, confirming the data was accessible and the finding was not a false positive. This is the filter that converts machine signal into actionable intelligence.

What does a scanner actually show vs. what we found?

The standard approach to external security testing is to run a vulnerability scanner against your domain. Scanners enumerate known CVEs, check headers, test for common misconfigurations, and produce a list of findings ranked by severity. They are fast, cheap, and broadly useful as a first pass. They also missed the majority of what we found in this study.

IDOR vulnerabilities - the largest category of confirmed findings - are almost entirely invisible to scanners. A scanner does not understand that a user token associated with Account A should not be able to access the records of Account B. It just sees a valid HTTP request returning a 200 response. The hardcoded AWS secret in a client-side bundle? A scanner focused on headers and CVEs will not catch that. The unauthenticated admin endpoint? Only if it matches a known pattern in the scanner’s ruleset. Greywatch's AI agents probe actual behavior, not pattern matches.

Why are AI-generated codebases specifically vulnerable?

A significant portion of the products we tested were built with substantial AI code generation - Cursor, Copilot, and similar tools are now part of the standard startup development workflow. AI-generated code is optimized for one question: does it work? It produces correct-looking, functional code that passes tests and ships features. It does not consistently apply security controls, because security controls are often invisible to the function the code is trying to perform. An AI generating a user profile endpoint will return the user object. It will not necessarily add the ownership check that verifies the requesting user is the owner of that profile.

This matters at scale because AI code generation accelerates the surface area problem. A team of three engineers can now produce the codebase of a team of twenty. That codebase has twenty engineers’ worth of endpoints, data models, and API interactions - and three engineers’ worth of security review. The agents find the gap between what was built and what was secured. In AI-assisted codebases, that gap is wider and it grows faster.

What does this mean for your product?

The three most severe individual findings in this study illustrate what is actually at stake:

• IDOR account takeover - an attacker could take control of any user account on the platform by changing a single integer in the request. No credentials required. Full session access.
• Unauthenticated database endpoint - an internal API endpoint left publicly accessible returned the full contents of a user table including names, emails, and hashed passwords with no authentication required.
• AWS credentials in client bundle - a production AWS secret key was embedded in the compiled JavaScript bundle served to every visitor. The key had broad S3 and DynamoDB permissions.

If you hold customer data and ship code regularly, the question is not whether you have vulnerabilities. The study suggests you probably do. The question is whether you find them before an attacker does, and whether you can fix them fast enough to matter.

How Greywatch runs this on your product

What we ran against these 30 startups are the same AI agents we run against our customers on a continuous basis. The difference is that for customers, the agents run as often as they ship code, findings are contextualized against their specific product, and every confirmed vulnerability ships with the exact request, the data exposed, and a recommended fix.