What Penetration Testing Companies Mean When They Say 'Methodology': PTES, OSSTMM, NIST 800-115, and How to Compare Them as a Buyer

Why Penetration Testing Methodology Comparison Matters for Buyers

Almost every penetration testing company lists a methodology on its website. Most mention PTES, OSSTMM, NIST 800-115, or the OWASP Testing Guide. Some claim a proprietary hybrid approach. For buyers evaluating multiple firms, a practical penetration testing methodology comparison is essential because the framework a company follows directly affects what gets tested, how findings are reported, and whether the engagement meets your compliance or risk management goals.

The problem is that these framework names often function as marketing shorthand. A company can write “PTES-aligned” on a landing page without meaningfully following the Penetration Testing Execution Standard. Buyers rarely have the context to distinguish between genuine adherence and name-dropping.

This article breaks down each major framework, explains what it covers and what it does not, and gives you a concrete set of questions for procurement conversations.

The Major Frameworks, Explained

PTES (Penetration Testing Execution Standard)

PTES defines seven phases of a penetration test:

1. Pre-engagement interactions
2. Intelligence gathering
3. Threat modeling
4. Vulnerability analysis
5. Exploitation
6. Post-exploitation
7. Reporting

PTES is the most commonly cited framework among commercial pentest firms. Its strength is its operational structure. Each phase has defined objectives and expected outputs. The pre-engagement section covers scoping, rules of engagement, and legal considerations. The post-exploitation section addresses pivoting, data exfiltration, and persistence, areas that lighter assessments often skip.

What PTES emphasizes: Full attack simulation from reconnaissance through post-exploitation. Clear phase separation. Detailed guidance on what testers should do at each stage.

What PTES leaves out: It does not define specific compliance mappings. It does not prescribe quantitative metrics for measuring security posture. It says little about continuous testing or retesting cadence. PTES was last substantially updated years ago, and some of its technical guidance (particularly around tooling) is dated.

Best fit: General-purpose network and infrastructure penetration tests. Organizations that want a structured, attack-simulation-style engagement.

OSSTMM (Open Source Security Testing Methodology Manual)

OSSTMM, maintained by ISECOM, takes a different approach. It defines security testing across five channels: human, physical, wireless, telecommunications, and data networks. Its distinguishing feature is the RAV (Risk Assessment Values) system, which attempts to produce a quantitative security metric called the “attack surface” score.

What OSSTMM emphasizes: Measurable outputs. Operational security as a concept broader than just technical testing. Testing across non-digital channels. Consistency and repeatability.

What OSSTMM leaves out: It is less prescriptive about specific exploitation techniques. Its quantitative model, while intellectually interesting, requires significant effort to implement correctly and is not widely adopted in commercial engagements. OSSTMM’s documentation can be dense, and fewer testers have practical experience with it compared to PTES.

Best fit: Organizations that want metrics-driven security assessments. Engagements that span physical and digital security. Companies that need to benchmark security posture over time.

NIST SP 800-115 (Technical Guide to Information Security Testing and Assessment)

NIST 800-115 is a publication from the National Institute of Standards and Technology. It sits within the broader NIST Special Publication 800 series, which means it is designed with federal information systems in mind but is used well beyond government contexts.

NIST 800-115 defines three types of testing: testing (hands-on, technical), examination (reviewing documentation, logs, configurations), and interviewing (talking to personnel). It covers penetration testing as one component of a broader security assessment program.

What NIST 800-115 emphasizes: Integration with organizational risk management. Structured planning and ROE (rules of engagement) documentation. Post-test analysis and remediation tracking. Alignment with other NIST frameworks (800-53, the RMF).

What NIST 800-115 leaves out: Deep technical guidance on exploitation techniques. It is a higher-level document than PTES or OSSTMM. It does not provide phase-by-phase tester playbooks. Companies that claim NIST 800-115 as their methodology may be using it more as a governance wrapper than an operational testing guide.

Best fit: Federal agencies and federal contractors. Organizations subject to FISMA. Any environment where NIST compliance is already a requirement.

OWASP Testing Guide

The OWASP Testing Guide is specific to web application security. The current version (v4.2) defines 91 test cases organized into 12 categories, including information gathering, configuration management, identity management, authentication, authorization, session management, input validation, error handling, cryptography, business logic, and client-side testing.

What OWASP Testing Guide emphasizes: Comprehensive web application test coverage. Specific, repeatable test cases. Alignment with the OWASP Top 10 but going well beyond it. Detailed descriptions of how to test for each vulnerability category.

What OWASP Testing Guide leaves out: It is not a full engagement methodology. It does not cover pre-engagement scoping, network-level testing, post-exploitation, or reporting standards. Companies typically use it as a technical reference within a broader engagement framework.

Best fit: Web application penetration tests. API security assessments. Organizations that need to verify OWASP Top 10 coverage and go deeper.

Proprietary and Hybrid Methodologies

Many pentest companies describe a “proprietary methodology” or a hybrid that combines elements from multiple frameworks. This is common and not inherently a problem. Experienced firms often develop internal processes refined through hundreds of engagements.

The risk is when “proprietary methodology” is used to avoid specificity. A legitimate proprietary approach should be describable in concrete terms: phases, deliverables, tool selection criteria, escalation procedures, and reporting structure.

Side-by-Side Comparison

Attribute	PTES	OSSTMM	NIST 800-115	OWASP Testing Guide
Scope	Network, infrastructure, general purpose	Multi-channel (human, physical, wireless, telecom, data)	Information security assessment broadly	Web applications and APIs
Phase structure	7 defined phases	Channel-based testing modules	3 activity types (test, examine, interview)	12 test categories, 91 test cases
Quantitative metrics	No	Yes (RAV scores)	No	No
Compliance mapping	No	No	Yes (NIST 800-53, RMF)	Partial (OWASP Top 10)
Post-exploitation guidance	Yes, detailed	Limited	General	No
Reporting standards	Defined but flexible	Defined with metric outputs	Defined, emphasizes remediation tracking	Not defined (test-case-level only)
Commercial adoption	High	Low to moderate	Moderate (high in government)	High for web app testing
Last major update	2014 (technical guidelines)	Version 3 (2010)	2008 (with ongoing NIST context updates)	Version 4.2 (2020)

One pattern worth noting: none of these frameworks are new. Their age does not make them irrelevant, but it means testers must supplement them with current techniques, particularly for cloud environments, containerized infrastructure, and modern API architectures.

What Companies Actually Do vs. What They Claim

Here is a common scenario. You send an RFP to five pentest firms. Three mention PTES. One mentions OSSTMM. One describes a proprietary methodology. During scoping calls, you ask each company to walk you through their process.

What you will often find:

• Some companies can describe their methodology in detail, phase by phase, with specifics about tooling, communication cadence, and deliverable formats.
• Some companies give vague answers like “we follow industry best practices” or “our methodology is based on PTES” without being able to describe what that means operationally.
• Some companies confuse methodology with tooling. They describe running Nmap, Burp Suite, and Metasploit, but cannot articulate a structured testing process around those tools.

A company that cannot describe its methodology during a live conversation probably does not follow one in a disciplined way. Tools are not a methodology. Running a scanner is not penetration testing.

A Buyer’s Checklist for Evaluating Methodology Claims

Use these questions during scoping calls or in your RFP. They are designed to separate firms with genuine methodological rigor from those using framework names as decoration.

Questions About Process

• “Walk me through your engagement phases from scoping to final deliverable.” A competent firm will describe a clear sequence: pre-engagement, reconnaissance, testing, reporting, remediation support. Listen for specifics, not buzzwords.

• “How do you handle scope changes mid-engagement?” This tests operational maturity. Methodology frameworks define initial scope, but real engagements often require adjustments. The company should have a documented change process.

• “What does your reconnaissance phase produce?” Ask for concrete outputs. A PTES-aligned firm should describe intelligence gathering deliverables: network maps, technology fingerprinting, OSINT findings.

Questions About Reporting

• “Can I see a sample report?” Most firms provide redacted samples. The report structure tells you a lot about methodology. Look for executive summary, methodology description, finding severity ratings with a defined scale, reproduction steps, and remediation guidance.

• “How do you classify finding severity?” Ask what rating system they use. CVSS-based? Custom? A firm following a defined methodology should have a consistent, documented approach to severity classification.

Questions About Depth

• “Do you perform post-exploitation testing? What does that include?” This separates vulnerability assessments from genuine penetration tests. Post-exploitation (lateral movement, privilege escalation, data access demonstration) is where real risk becomes visible.

• “How do you handle situations where exploitation could impact production systems?” This tests whether the firm has mature rules of engagement and communication protocols. Methodology frameworks address this, and so should the company.

Questions About Compliance Alignment

• “How does your methodology map to our compliance requirements?” If you need PCI DSS, SOC 2, HIPAA, or FedRAMP alignment, the company should be able to explain how their testing process satisfies the relevant controls. Not all methodologies map cleanly to all compliance frameworks.

How Methodology Affects What You Get

The methodology a company follows (or does not follow) has downstream effects on your engagement:

Scoping accuracy. Frameworks with defined pre-engagement phases (PTES, NIST 800-115) tend to produce more accurate scoping. This means fewer surprises on timeline and cost.

Finding quality. A methodology with structured exploitation and post-exploitation phases will typically surface deeper findings than one focused only on vulnerability identification.

Report usefulness. Methodology-driven reports follow a consistent structure. This matters when you are comparing results across multiple engagements or tracking remediation progress over time.

Repeatability. If you engage the same firm (or a different firm) for retesting, a defined methodology makes it possible to compare results consistently. Without one, you are comparing apples to estimates.

Red Flags During Procurement

Watch for these signals that a company’s methodology claims may not hold up:

• They cannot name their methodology or describe it beyond a framework name. “We use PTES” is a starting point, not an answer.
• They conflate automated scanning with penetration testing. Running Nessus is not a methodology.
• Their sample report lacks a methodology section. A firm that follows a defined process will document it in every report.
• They cannot explain how they handle the phases they did not perform. If they skip post-exploitation, they should be able to articulate why and what that means for finding completeness.
• They resist providing a sample report. This is standard practice. Resistance suggests the report may not reflect a structured process.

Using pentest.fyi to Compare Companies by Methodology

When you search for penetration testing companies on pentest.fyi, you can filter and compare firms based on their stated services, certifications, and testing approaches. This makes it easier to build a shortlist of companies that align with your methodology requirements before you start scoping calls.

Look at each company’s listing for mentions of specific frameworks. Then use the questions above during your evaluation. The directory gives you breadth; the scoping call gives you depth.

Choosing a Framework Requirement for Your RFP

Should you specify a methodology in your RFP? It depends on your situation.

Specify a framework when:

• Your compliance requirements mandate it (e.g., NIST 800-115 for federal systems).
• You need to compare results across multiple vendors over time and need consistency.
• Your internal security team has experience with a specific framework and wants alignment.

Specify outcomes instead when:

• You are buying penetration testing for the first time and are still learning what you need.
• Your scope crosses multiple domains (web, network, cloud, physical) and no single framework covers everything.
• You want to evaluate the company’s own methodology as a signal of their maturity.

A practical middle ground: describe your scope and compliance requirements in the RFP, ask respondents to identify their methodology, and then evaluate their answers using the checklist above.

Final Perspective

Methodology names on a website are a starting point. They tell you what a company aspires to, not necessarily what it delivers. The real test happens during the scoping call, in the sample report, and in the engagement itself.

Buyers who ask specific, informed questions about methodology will consistently select better providers. The frameworks themselves (PTES, OSSTMM, NIST 800-115, OWASP) each have strengths and limitations. None of them are complete on their own. What matters most is whether the company you hire has internalized a structured process and can execute it with discipline.

Use the comparison table and checklist in this article as a reference during your next procurement cycle. And use pentest.fyi to build your initial list of companies to evaluate.

Key Takeaways

• Most penetration testing companies cite a methodology framework on their website, but the practical differences between PTES, OSSTMM, NIST 800-115, and OWASP matter significantly for scoping and deliverables.
• OSSTMM is the only major framework that defines quantitative security metrics, while PTES focuses on operational phases and NIST 800-115 emphasizes compliance context.
• A company that cannot walk you through its methodology during a scoping call is a meaningful red flag, regardless of which framework it claims.
• Proprietary or hybrid methodologies are not inherently worse than named frameworks, but they require more scrutiny during evaluation.
• Asking five or six specific questions about methodology during procurement will separate competent firms from those using framework names as marketing copy.

Frequently Asked Questions (FAQ)

Which penetration testing methodology is the best?

There is no single best framework. PTES suits general-purpose engagements. OSSTMM works well when you need measurable security metrics. NIST 800-115 fits federal or compliance-driven environments. OWASP Testing Guide is the standard for web application assessments. The right choice depends on your scope, compliance requirements, and what you intend to do with the results.

What does it mean when a pentest company says they use a proprietary methodology?

It means they have developed their own process, often combining elements of public frameworks with internal playbooks. This can be perfectly valid if they can describe it in detail. Ask them to walk you through their phases, how they handle scope changes, and what their reporting structure looks like.

Should I require a specific methodology in my RFP?

Only if your compliance requirements mandate it. Otherwise, specify the outcomes you need (coverage areas, reporting format, retest policy) and ask respondents to describe how their methodology achieves those outcomes. This gives you better signal than requiring a framework name.

How can I tell if a company actually follows the methodology it claims?

Ask specific questions during the scoping call. Request a sample report. Ask how they handle the intelligence gathering or reconnaissance phase. Ask what tools they use at each stage. Companies that genuinely follow a methodology can answer these questions without hesitation.

Is OWASP Testing Guide a penetration testing methodology?

It is a testing guide specifically for web application security. It defines test cases and categories rather than engagement phases. Most companies use it alongside a broader methodology like PTES when the scope includes web applications.