Penetration Testing

Penetration testing is human-driven, adversarial testing of a system's defenses — distinct from vulnerability scanning, which checks for known signatures at scale but doesn't chain findings together, exploit business logic flaws, or think like an attacker. Scanning tells you a door might be unlocked; a penetration test actually tries the handle, and then checks what's reachable once it's open. This article covers what should trigger a test, the different approaches available, and how to run one so it produces real risk reduction rather than a report that sits in a drive.

What Triggers a Penetration Test

Treat penetration testing as driven by defined triggers, not an occasional reactive purchase. Two categories of trigger matter most:

1. Regular, Independent, Compliance-Driven Testing

Driver	Typical Requirement
PCI DSS	Requirement 11.4: annual penetration test (internal and external), plus after any significant change; segmentation testing at least every 6 months if segmentation is used to reduce PCI scope
SOC 2 Type II	Not mandated by the criteria itself, but routinely required by auditors and enterprise customers as evidence supporting the Security trust criterion
ISO/IEC 27001	Annex A controls on security testing and independent review of information security (A.8.29, A.5.35/5.36 in the 2022 revision) commonly satisfied via periodic independent pen testing
APRA CPS 234	Requires regular testing of security control effectiveness, with testing independence from the team that designed/implemented the control — see Frameworks Overview
Cyber insurance	Many policies now condition coverage (or premium tier) on an annual independent penetration test
Regulatory examination follow-up	Findings from a regulator or auditor frequently require independent validation that remediation actually closed the gap

The word independent is load-bearing in most of these — the team that built or maintains a system generally shouldn't be the one attesting to its security via the same test used for compliance evidence. Use an accredited third party, or an internal team with genuine organizational separation, for anything feeding a compliance obligation.

2. Significant Architectural or System Changes

Launch of a new internet-facing application or service.
Major infrastructure change — cloud migration, network re-architecture, new data center, significant identity/access model change.
Mergers and acquisitions bringing new systems, networks, or codebases into scope.
Significant changes to authentication, authorization, or payment-processing components specifically.
A new third-party integration with elevated data access (see MCP Security Assessment for the AI-agent-specific version of this trigger).
Post-incident, to validate that remediation of a breach or significant security event actually closed the exploited path — not just the specific symptom.
Ahead of a major public launch or marketing push that will materially increase traffic and attacker attention.

A useful operating rule: if a change would require re-answering "what can an attacker reach, and with what privilege" differently than before, it's a trigger — regardless of whether a compliance calendar happens to call for a test that quarter.

Types of Penetration Testing

By Knowledge Level

Type	Tester Starts With	Strength	Tradeoff
Black-box	No prior knowledge — simulates an external attacker from scratch	Most realistic simulation of an opportunistic external attacker	Time-boxed engagements may not reach deeper issues reconnaissance alone can't surface
White-box	Full knowledge — source code, architecture diagrams, credentials	Deepest, most thorough coverage in a fixed time window	Less representative of what a real external attacker actually starts with
Grey-box	Partial knowledge — e.g., a standard user account, high-level architecture	Balances realism with depth; the most common real-world choice	Requires deciding up front how much access is "realistic" for the threat model being tested

By Target

Network penetration testing — external (internet-facing) and internal (assumed-breach/insider simulation) network infrastructure.
Web application penetration testing — the OWASP Top 10 and beyond, business logic, authorization boundaries.
API penetration testing — authentication, object-level authorization (BOLA/IDOR), rate limiting, mass assignment — see API Security.
Mobile application penetration testing — client-side storage, certificate pinning, platform-specific misconfigurations, backend API abuse via the mobile client.
Cloud configuration/penetration testing — IAM misconfigurations, storage exposure (see Securing Cloud Storage Buckets), metadata service abuse, cross-account trust issues.
Wireless penetration testing — rogue access points, WPA weaknesses, network segmentation from the wireless side.
Physical penetration testing — badge cloning, tailgating, unsecured network jacks.
Social engineering / phishing assessments — the human layer, often run alongside a technical test rather than as a substitute for one.

Red Team vs. Standard Penetration Test

A standard penetration test is typically broad-coverage and time-boxed: find and report as many exploitable issues as possible within scope. A red team engagement is goal-oriented and narrow: achieve a specific objective (e.g., access to a crown-jewel system) using any combination of technical, physical, and social vectors, while deliberately evading detection — testing people and process (does the SOC notice?) as much as technology. Don't substitute one for the other; they answer different questions.

Methodology and Standards

Rather than inventing a process from scratch, anchor engagements to an established methodology:

PTES (Penetration Testing Execution Standard) — seven phases: pre-engagement, intelligence gathering, threat modeling, vulnerability analysis, exploitation, post-exploitation, reporting.
OWASP Web Security Testing Guide (WSTG) — the standard reference for web application test coverage.
NIST SP 800-115 — Technical Guide to Information Security Testing and Assessment; useful for structuring engagements against a recognized federal standard.
OSSTMM — Open Source Security Testing Methodology Manual, with a strong focus on measurable, repeatable test rigor.
MITRE ATT&CK — not a testing methodology itself, but the standard reference for real-world adversary techniques when scoping a red team/adversary-emulation engagement.

Guidelines for Running an Effective Engagement

Define scope and Rules of Engagement (RoE) in writing before anything starts — in-scope and explicitly out-of-scope assets, the testing window, permitted techniques (is DoS testing allowed? social engineering? production data modification?), emergency contacts, and how any sensitive data encountered during testing will be handled and disposed of afterward.
Get written authorization before testing begins. For cloud-hosted assets, check the specific cloud provider's penetration testing policy first — AWS, Azure, and GCP each publish their own rules on what's pre-authorized versus what requires notification, and some techniques (e.g., DNS zone walking, volumetric DoS testing) are typically prohibited outright regardless of provider.
Scope risk-based, not alphabetically — prioritize internet-facing systems and anything handling regulated or high-value data first, rather than working through an asset inventory in whatever order it happens to be listed.
Match the knowledge level to the actual objective. Compliance-driven testing often defaults to black-box; a genuine assurance objective for a complex application usually benefits more from grey- or white-box access in the time available.
Use genuinely independent testers, per the compliance-driver discussion above — and verify tester competency (OSCP, OSCE, CREST, GPEN, or equivalent demonstrated experience) rather than taking a vendor's marketing at face value.
Combine automated tooling with manual testing. Automated scanners (Burp Suite, Nessus, Nuclei) efficiently catch known patterns and misconfigurations; they routinely miss business logic flaws, authorization bypass, and multi-step exploit chains that only a human tester connects.
Test against an environment that reflects production. Testing a stale lower environment with different configuration, data volume, or patch level produces false assurance — either test production carefully within agreed guardrails, or keep a staging environment in genuine parity.
Establish safe-testing practices up front — explicit agreement on what's off-limits for exploitation (e.g., confirm-only for anything that could cause outage or data loss), a communication channel for the testing team to flag anything urgent immediately, and a rollback plan if something does go wrong.
Validate every finding manually. A report should contain proof-of-concept evidence and reproduction steps, not raw, unfiltered scanner output — false positives erode trust in the whole report and waste remediation effort.
Prioritize by actual exploitability and business impact, not raw CVSS alone. A moderate-severity finding on an internet-facing system chained with an internal one can be more urgent than a standalone "critical" that requires implausible preconditions to reach.
Deliver reporting for two audiences — an executive summary framing risk and business impact for leadership, and full technical detail (steps to reproduce, affected components, remediation guidance) for the engineers who have to fix it.
Track remediation to closure, and retest. A finding that's fixed but never retested is an assumption, not a verified control — close the loop with a targeted retest of previously identified issues.
Treat this as point-in-time, not continuous. A penetration test validates a snapshot; it doesn't replace continuous controls like SAST and SCA, DAST in CI/CD, or ongoing vulnerability scanning — it complements them.
Decide deliberately whether to notify your SOC/security operations team ahead of time. Notifying them avoids wasted incident-response effort chasing the test itself; deliberately not notifying them is the correct choice when the objective specifically includes testing detection and response capability (i.e., a red team engagement).

Roles and Independence

Whoever scopes and executes the test should not be the same individuals responsible for designing or maintaining the system under test — the same principle CPS 234-style regulatory testing requirements codify explicitly. For compliance-driven engagements specifically, prefer an accredited external firm; for maturity-building internal red/purple team programs, ensure genuine reporting-line separation from the teams being tested even if both sit inside the same security organization.

Engagement Checklist

Item	Status
Trigger identified and documented (compliance cycle, architecture change, or other)	☐
Scope and Rules of Engagement agreed and signed off in writing	☐
Written authorization obtained; cloud provider policy checked if applicable	☐
Tester independence and competency verified	☐
Knowledge level (black/grey/white-box) matched to the objective	☐
Safe-testing guardrails and emergency contacts agreed	☐
SOC notification decision made deliberately, not by default	☐
Findings manually validated with proof-of-concept evidence	☐
Report delivered with both executive and technical detail	☐
Remediation tracked to closure with a scheduled retest	☐
Next trigger/cadence scheduled rather than left implicit	☐

NIST SP 800-53 Rev. 5 and Framework Mapping

Control / Reference	Title	Applied Here
CA-8	Penetration Testing	The direct NIST control covering this entire practice
RA-5	Vulnerability Monitoring and Scanning	The complementary, continuous counterpart to point-in-time pen testing
SA-11	Developer Testing and Evaluation	Pre-release security testing feeding into broader assurance
PM-14	Testing, Training, and Monitoring	Organizational-level commitment to a testing program, not one-off tests
PCI DSS Requirement 11.4	Penetration testing	Explicit annual/change-triggered requirement for cardholder data environments
CIS Controls v8, Control 18	Penetration Testing	Prioritized, actionable control guidance complementing this framework mapping