Request a Demo
AI AND LLM PENTEST

Secure the AI you are building

Cobalt finds and confirms the vulnerabilities that matter in the AI and LLM-based applications your team is shipping. Release new AI features with confidence, and remediate LLM exploits with clarity.
THE CHALLENGE

AI applications carry risk traditional testing was not built to find

AI and LLM applications move data and make decisions in ways conventional applications do not, and the failure modes follow the model, not the perimeter.


A few examples include:
  • Prompt injection overrides the instructions you set.
  • Indirect injection arrives through the documents your application retrieves.
  • Sensitive data leaks through outputs, embeddings, and system prompts.
  • Excessive agency turns a helpful assistant into an attacker's tool.

Your teams are releasing AI features faster than traditional testing can keep up. Scanners cannot reason about any of these AI related issues, and the teams shipping AI fastest are the ones carrying the most untested exposure.

OUR APPROACH

A field-tested AI methodology, executed by the Cobalt Core

Cobalt follows an industry-leading methodology informed by the OWASP Top 10 for Large Language Model Applications, refined over three years of AI and LLM testing at scale. Human-led testing goes where automation cannot, probing how your model handles adversarial input, and AI accelerates reconnaissance and triage so pentesters spend their time on exploitation and business logic. Cobalt pentesters help write the standards your auditors and customers are asking about, contributing directly to the OWASP Top 10 for LLM Applications.

Explore AI & LLM pentest methodologies

What a Cobalt AI and LLM pentest covers

map-light-full

Map the application and its attack surface

Pentesters gather information the way a malicious user would, identifying input vectors, how outputs are used, integrated systems, and third-party dependencies, then scope the test to how your application actually works.
bullseye-arrow-light-full

Probe the model's behavior and boundaries

Testers send crafted inputs to learn the model's implicit rules, the sanitization applied to inputs and outputs, and where its guardrails give way, using manual interaction, prompt engineering frameworks, and HTTP proxies.
shield-exclamation-light-full

Identify and exploit across the OWASP LLM Top 10

Expert pentesters confirm what is actually exploitable, from prompt injection, sensitive information disclosure, and system prompt leakage to improper output handling, excessive agency, and unbounded consumption.
diagram-project-sharp-light-full

Test the retrieval layer, not just the chat interface

For applications that retrieve from internal data, pentesters test the retrieval pipeline, document and data-source injection, vector and embedding weaknesses, and access controls on retrieved content.

Find and fix critical AI and LLM vulnerabilities

Every finding includes steps to reproduce and remediation guidance, all reported in real time and available for integration into your existing tooling and workflows through the Cobalt Platform.
Example AI Finding


AI Application Pentesting
AI pentests assess application security, agentic workflows, access controls, and backend integrations for vulnerabilities that could expose sensitive data or enable unauthorized actions.

CRITICAL (CVSS 9.6): End-User AWS Role Grants Unauthorized Access to Server-Side Secrets and Agentic Framework Configuration

Severity: Critical | CVSS: 9.6 | CWE: CWE-284 (Improper Access Control) | OWASP LLM Top 10: LLM06: Excessive Agency | Status: Pending Fix

Description

The AI chat application vends short-lived AWS credentials to authenticated end-user browser sessions via a Cognito identity pool. These credentials are scoped to a single IAM role intended only for client-side operations. In practice, the role's IAM policy grants significantly broader permissions: it allows reading and decrypting internal SSM Parameter Store entries under a server-side configuration path, retrieving live Secrets Manager secrets containing machine-to-machine OAuth credentials for the agentic backend, retrieving an analytics signing key bundle from a second Secrets Manager secret, and listing and reading internal agentic framework configuration files from an S3 bucket path intended for backend use only.

A standard authenticated employee session was used to obtain the Cognito-vended AWS credentials. Using only the AWS CLI configured with those credentials, the following server-side assets were fully accessible without any additional privilege escalation.

Affected Resources

  • [llm-chat-api-domain] — AI chat application API backend
  • SSM Parameter Store path: /chat/v3/uat/server/* — server-side configuration including client secrets
  • Secrets Manager secret: uat/agentic-chat/cognito-m2m-credential-details — M2M OAuth credentials
  • Secrets Manager secret: chat-v3-uat-secrets-analytics — analytics signing key bundle
  • S3 path: s3://[chat-s3-bucket]/config/agenticframework/adminpro/ — agentic framework configuration

Proof of Concept

Step 1: Authenticate as any standard user and intercept the response containing the Cognito-vended AWS credentials (AccessKeyId, SecretAccessKey, SessionToken).

Step 2: Configure the AWS CLI with the obtained credentials:

  1. export AWS_ACCESS_KEY_ID=$ACCESS_KEY_ID
  2. export AWS_SECRET_ACCESS_KEY=$SECRET_ACCESS_KEY
  3. export AWS_SESSION_TOKEN=$SESSION_TOKEN
  4. export AWS_DEFAULT_REGION=us-east-1

Step 3: List and read internal agentic framework configuration files from the S3 backend path:

  1. aws s3 ls s3://[chat-s3-bucket]/config/agenticframework/adminpro --recursive
  2. # Returns:
  3. config/agenticframework/adminpro/agent-configs.yaml
  4. config/agenticframework/adminpro/tool-configs.yaml
  5. config/agenticframework/templates/agent-template.yaml

Files contain agent prompt content, tool wiring configuration, model identifiers, and internal backend service endpoint references.

Step 4: Retrieve server-side client secrets from SSM Parameter Store:

  1. aws ssm get-parameters-by-path --path /chat/v3/uat/server --with-decryption --recursive
  2. # Returns plaintext values for:
  3. # [client-api-secret-1], [client-portal-secret-1],
  4. # [client-llm-response-secret-1], [client-llm-anonymizer-secret-1],
  5. # [client-llm-agent-secret-1], jwt-secret

Step 5: Retrieve the M2M OAuth credential secret from Secrets Manager:

  1. aws secretsmanager get-secret-value --secret-id uat/agentic-chat/cognito-m2m-credential-details
  2. # Returns: client_id, client_secret, token_endpoint, scope
  3. # (machine-to-machine credentials used by the agentic backend)

Impact

A standard authenticated user obtains all server-side secrets and internal agent configuration in a single AWS CLI session. Exposed material includes active M2M OAuth credentials sufficient to impersonate the agentic backend, multiple API client secrets, a JWT signing key, an analytics signing key bundle with active key metadata, and the complete agent prompt, tool wiring, and configuration for the internal AI agent.

Possession of these secrets enables an attacker to impersonate backend services, forge authenticated API calls, extract the agent's internal instructions and tool definitions, and potentially manipulate the agentic framework's behavior at the configuration level.

Suggested Fix

  • Remove all SSM Parameter Store, Secrets Manager, and internal S3 configuration access from the end-user Cognito IAM role. These resources must only be accessible to backend service roles.
  • Split roles cleanly: the browser/end-user role should permit only the minimal client operations required for the chat UI; all secret and internal config access must reside in a separate backend service role that is never vended to browser clients.
  • Move internal agent prompts, tool configurations, and templates to a separate S3 prefix or bucket that is not included in the end-user role's resource scope.
  • Rotate all exposed secrets immediately: revoke and reissue all client secrets, M2M credentials, and signing keys that were accessible via the end-user role.
  • Audit CloudTrail for any historical access to the affected SSM, Secrets Manager, and S3 paths using Cognito-vended credentials to determine whether exploitation has already occurred.

Anonymization Notes

  • Client domain names and hostnames removed or replaced with [target-domain], [api-target-domain], [web-target-domain], [mobile-api-target-domain], or [llm-chat-api-domain].
  • Product code names replaced with generic descriptions (e.g., banking application, iOS application, CMS module, reservation assistant, AI chat application).
  • AWS account IDs, CodeBuild project names, IAM user names, S3 bucket names, and SSM paths replaced with generic placeholders.
  • Azure AD tenant domain replaced with [tenant-domain]; application display names retained where they identify Microsoft tooling.
  • Application-specific session cookie names and internal tokens anonymized.
  • Test credentials replaced with [test-user], [password], or <REDACTED_API_KEY> placeholders.
  • Third-party SaaS platform names and domains replaced with [support-platform-domain].
  • ColdFusion target domains replaced with [target-domain] — IP addresses removed.
  • AI/LLM chat interface URLs replaced with [chat-demo-url].
  • Internal Active Directory hostnames replaced with [internal-ca-host].
  • MongoDB and GraphQL target domains replaced with [api-target-domain] and [mobile-api-target-domain].
  • SSM Parameter Store secret names containing client identifiers replaced with generic placeholders.
WHAT YOU GET

Testing scoped to how your AI application is built

Prompt Injection

Neutralize the most common AI threats with rapid testing that prevents prompt leakage and instruction bypass. Prompt injection testing is best for basic chat interfaces.

Isolated LLM

Prevent malicious users from manipulating your AI’s behavior or forcing it to ignore safety settings. Isolated LLM testing is best for standalone LLM applications.

RAG-Enabled LLM

Secure your data integrity by ensuring the AI only provides the right data to the right user—and rebuffs attacks. RAG-Enabled LLM testing is best for complex LLMs connected to proprietary data.
CUSTOMER TESTIMONIALS

Trusted to test AI where it counts

PowerSchool Logo, small
There is a lot of snake oil out there. We loved that Cobalt was the real thing and actually knew what they were talking about. They were the first pentesting solution that presented us with a clear methodology on AI and LLM applications.
Read full story

Mishka McCowan

Chief Information Security Officer, PowerSchool
industry-vertical-zest-logo-cropped
We've been working with Cobalt for years before they started offering LLM pentesting. We knew Cobalt was committed to staying at the forefront of industry best practices, so when we started developing LuLu, it was a no-brainer to continue to work with them.
Read full story

Waylan Wong

Cloud Engineering, Sr. Manager, Zest AI
PowerSchool Logo, small
AI security isn't a one and done process. Continuous testing is essential to stay ahead of evolving risks, and Cobalt's expertise makes them the ideal partner for this ongoing effort.
Read full story

Mishka McCowan

Chief Information Security Officer, PowerSchool
FAQ

Frequently asked questions about AI and LLM pentesting

What is AI and LLM penetration testing?
AI and LLM penetration testing is a controlled, authorized assessment of an AI-enabled application to find and confirm exploitable vulnerabilities before real attackers do. Cobalt combines human-led testing with AI-powered automation to evaluate how your model handles adversarial input, protects sensitive data, and interacts with connected systems and retrieval sources.
What methodology does Cobalt use for AI and LLM pentesting?
Cobalt follows an industry-leading methodology informed by the OWASP Top 10 for Large Language Model Applications, and refined over three years of AI and LLM testing. Testing spans application reconnaissance and scope definition, interactive probing and system understanding, vulnerability identification and exploitation across the OWASP LLM Top 10, and real-time reporting, triaging, and retesting.
What can Cobalt test, and at what levels?
Cobalt offers three levels of AI and LLM testing. A Prompt Injection pentest is a focused, agile test of prompt injection resistance. An Isolated LLM pentest covers a standalone application and the web and API components that interact with it. A RAG-Enabled LLM pentest extends that coverage to applications that retrieve from external knowledge sources, adding the retrieval pipeline, vector and embedding weaknesses, and access controls on retrieved content.
What kinds of vulnerabilities does an AI and LLM pentest find?
Common findings include prompt injection and indirect injection, sensitive information disclosure, system prompt leakage, improper output handling such as cross-site scripting through model output, excessive agency, data and model poisoning, vector and embedding weaknesses, and unbounded consumption. Every finding is validated as exploitable and delivered with reproduction steps and remediation guidance.
How is an AI pentest different from a scan?
Automated tools cannot reason about how a model interprets adversarial input, what it retrieves, or how its output is used downstream. Human-led pentesting uncovers the prompt injection, logic, and retrieval-layer vulnerabilities that scanners overlook, which matters because 32% of AI and LLM findings are high-risk, 2.7 times the rate across pentests overall.
What does AI and LLM pentesting work well with?
AI and LLM pentesting pairs naturally with Web Application Pentest and API Pentest for the applications and endpoints around your model, Secure Code Review for the code behind it, and DAST for continuous coverage between pentests.
RESOURCES

The latest from Cobalt

GET STARTED

You are shipping AI fast. Make sure it is tested for the risks that follow.

Secure your AI and LLM applications with AI-powered continuous offensive security testing grounded in human expertise from Cobalt. Access findings and remediate risk in real time through the Cobalt Offensive Security Platform.

product-screenshot-FPO2