IMPORTANT: This document is a reference guide to the testing preparation checklist (kickoff) that we send to a client and is not intended as a standalone resource.

Contact / Help

• If you are a current client seeking help, please reach out via the contact methods provided during scoping.
• If you are interested in services for your organization, please Contact Us.

Invoice

Two options for payment:

• Engagement was requested to be post-paid; there is no action needed at this time
• Engagement was requested to be pre-paid in whole or part using a pre-paid bucket/bundle of hours; the provided invoice will need to be paid before scheduling testing.
• More details: https://penconsultants.com/cybersecurityUnlimited

If not already, provide or confirm accounts payable contact(s) for receiving invoices and any special instructions

Client Contacts

No additional details available for this section.

Testing Overview

High-level reference: https://penconsultants.com/overview

Area of Testing

External and/or Internal Network, Web Application/API, Mobile App, Wireless, Cloud, Software, IoT, Physical, Red Teaming, Social Engineering, etc. More details: http://penconsultants.com/serviceComparisonMatrix

Testing Approach

Vulnerability Scanning, Vulnerability Assessment, Penetration Testing. More details: http://penconsultants.com/serviceComparisonMatrix

Tier of Testing

Basic Sub-1, Basic Sub-2, Standard, Premium. More details: http://penconsultants.com/pentestingTierComparison

Reporting Level

We provide various levels of reporting to reduce the total billable hours and/or increase testing thoroughness: https://penconsultants.com/reportingLevels. Anything other than “Standard Report” is less formal.

Timeline / Deadline

No additional details available for this section.

Testing Times

No additional details available for this section.

Previous Testing

Providing previous testing reports is good practice, and in some cases required by regulation, to ensure that mitigations and detections implemented after the last evaluation are properly retested.

NOTE: If PEN Consultants performed your previous testing, and assuming we received a copy of any previous reports at that time, simply answer “Yes” and “PEN Consultants has this” (respectively)

Scope

External

Unless your ISP requires exclusion, all assigned IPs are included in-scope, even if some are reserved. We find services exposed on these “unusable” IPs (ex., routed subnets) approximately 20% of the time, and about half of those are exposed client services (vs. ISP).

If a service is discovered on what is assumed to be a reserved network or broadcast IP (typically the lowest or highest address in a subnet), it should not be excluded. In fact, from a security perspective, it may be the most interesting IP in the range. These addresses are often overlooked in firewall rules and can inadvertently allow access.

Internal

Unless otherwise instructed, our default, authorized scope for internal network pentests is all RFC1918 ranges:

• 10.0.0.0/8
• 172.16.0.0/12
• 192.168.0.0/16

Host discovery is performed on all RFC1918 ranges by default, and then included in scope. This gives us thorough visibility, often revealing shadow IT and forgotten assets that carry real risk.

Optional: Having a list of CIDRs with known live hosts allows us to confirm our testing platform is able to reach the intended scope.

Other Scope

This will include anything that falls outside of a network or web app/API scope – applications, mobile apps, social engineering, red teaming, etc.

Not in Scope

If needed: List anything here that may fall under the given scope, but which needs to be avoided.

If needed: Are there items outside of the IP range, and/or under your domain(s) that we should specifically avoid or are prohibited?

Web/App: Email & SMS Service

If applicable: The app needs the ability to send email so that all features work: Account registration, password reset, user alerts, etc.. Testing will be limited or impossible for features that require email or SMS service if those services are not working.

Web/App: Sample Data

If applicable: Ensure ample sample data is loaded for all objects. Although we’ll add additional testing during testing, having a “real-looking” data set to start with will greatly increase testing thoroughness and effectiveness.

Web/App: Bank Numbers

If applicable: When testing an application involving payment processing, we will need one of the following (please choose one):

1. Test card or banking numbers – Example: Stripe’s 4242-4242-4242-4242
2. Authorization for additional budget – e.g., using real dollars through our card number(s)

Notes:

• These are typically test credit card numbers, but could also include bank account numbers.
• If we are required to use our/real card numbers (e.g., for transactions involving real money), even a minimum transaction of $1 could result in hundreds of dollars in billable charges.
• Without the ability to test transactions, any payment processing-related functionality (e.g., can the payment process be abused) or subsequent processes (e.g., profile creation after payment, transactions, invoices, etc.) will be severely limited. In practical terms, this means those features will not be covered by the testing.

Mission Critical Systems

• List any systems that have been affected by previous testing or scanning activities, and provide guidance such as “fragile”, “do not touch”, “light touch only”, etc..
• Indicate whether any of the following testing activities have caused issues in past engagements: host and port scanning (e.g., Masscan, Nmap), vulnerability scanning (e.g., Nessus, Burp), NTDS.dit extraction from a DC, password spraying, default credential checks on web services or network devices (e.g., network gear, BMCs, etc.), Kerberoasting, MITM6 IPv6, LLMNR/NBT-NS poisoning (e.g., Responder), DNS record injection, SMTP relaying, etc..
• Think of a system that, if it were to go down, would cause major issues in performing the day-to-day functions of your organization. Media systems, financial systems, HR databases, etc. Are there down-times and/or maintenance windows for these systems?

Traffic volume sent during testing

Our default rates, unless otherwise noted and scoped, are:

• Network tests: 25,000 packets/sec for Masscan, 6k for Nmap, 100 simultaneous hosts w/3 checks per for Nessus, and comparable speeds with other tools/processes
• Web tests: 10 threads of requests, no delays
• Cloud: Comparable speeds

Please confirm firewall rules, state tables, CPU and memory resources, web/FaaS services (ex. Lambda), etc., can handle these loads or if we need to re-scope the engagement.

MiTM w/potential user interaction

Is MiTM with potential user interaction allowed or forbidden?

Default: Allowed. In moderation, targeted, as part of network pentesting. Examples: unsolicited MFA pushes, SSDP/Fake UPnP devices appearing in network neighborhood, and other mostly passive MiTM type attacks.

Forbidden: Not recommended. Will result in a limitation added to the report as testing thoroughness will be impacted.

Other Notes

Include any other relevant notes related to scoping here.

Threat Detection & Blocking

General / Monitor Only Mode

• This section deals with what is known as scan & testing Interference.
• This includes any appliance, software, 3rd-party service, endpoint agent, etc. that attempts to identify and block or throttle traffic – External and Internal.
  • External examples: Firewalls, CloudFlare, F5, AWS WAF, Akamai, etc.
  • Internal examples: Internal firewalls, Darktrace, etc.
  • Endpoint examples: Endpoint firewalls, Crowdstrike, etc.
• Example of typical mitigation features: delayed response times (ex. rate-limiting), dropping packets which are deemed malicious/malformed (ex. IDS/IPS), temporarily blocking an IP address (ex. DoS or threat protections), blocking of malformed/malicious web requests (ex. WAF), issuing TCP resets or dynamic firewall blocks (ex. Darktrace, Vectra, etc.), etc.

Interference

Any attack mitigation type feature must be disabled/placed in alert-only mode, as to not cause false-negatives.

• It is generally not possible to test without changes to allow our test traffic. (Exception: adversarial red teaming)
• Penetration testing requires whitelisting.

Most common: Client able to make FW/IPS rule changes if/when needed – ex. Changing to “alert only” mode

Further reading: http://penconsultants.com/shields

Whitelisting

Whitelist the IP(s)/endpoint(s) provided in the kickoff document.

• This change should only be made for traffic from the source(s) given above.
• Depending on the firewall manufacturer/model, this may require specific firewall policies with IPS profile settings to properly exclude this source IP from inspection. In rare cases, this may not be possible without completely disabling the feature globally.
• This can be done now for our proxy IP, as we will NOT be coming from this IP address for any initial black box or red teaming, and will also come from a non-whitelisted IP at some point for all testing.

Active Services Found – External

These services are usually found with just a quick TCP top-1000 port scan. Sometimes, it may be a full TCP scan or even top-X UDP scan as well.

But generally, this would be 80%+ of what is ultimately found, so it is advised to confirm this aligns with what you expect to be exposed before testing begins. If the number is much lower than expected, it could be a sign of interference (ex., a firewall) or we have the wrong scope.

Scanning Interference Detected

If we detect a firewall rate limiting or blocking, a note will be made in this section.

If your firewall is found to be rate-limiting or blocking based on initial port scanning, it will generally not be possible to port scan the entire range within the given testing time without making firewall changes to prevent this. The port scanning and service enumeration phase must run to completion before testing can start. Without any needed firewall changes, this phase will take substantially longer than the total testing that was budgeted for, and all other phases of testing will take even more time.

WAF Interference Detected

Like with “Scanning Interference” above, any interference by a WAF will be noted.

For Network Testing, we can proceed despite WAF interference, but it is not advisable. For Wep App/API Testing, we cannot proceed unless our IP is whitelisted in any potential WAF.

Detection Alerts

Please monitor for and notify us of detections & alerts during testing.

• Option A: We will provide a list of activities at the end of testing and ask if you have seen alerts for various activities performed.
• Option B: Upload alerts as you receive them to the “FromClient” folder and we will cross-reference with our activity.
• Option C: Add the provided email address to your alerting distribution list so we automatically receive alerts.
• Option D: Provide us direct access to your logging/SIEM solution(s), and we can document what is seen in real-time, alleviating any burden on you.

Do you have an internal or external SOC?

This is intended to help think through whether there is anyone who should be notified of the upcoming testing, or maybe even have a contractual requirement for testing notification.

• If this is anything other than adversarial red teaming:
  • The testing WILL be noticed.
  • Appropriate parties must be prevented from mitigating traffic.
• Penetration testing will:
  • “Light up everything” – It is very loud.
  • Be impacted by blocks/active defenses – false negatives and higher costs.

Will your SOC be notified of testing?

• Option A (default approach): Start out without SOC/MSSP notification of testing to ensure they see the activity, but then immediately inform them at that time to prevent disruption.
• Option B: Client contacts who are aware of testing are the ones to process alerts, so they will handle them accordingly.
• Option C (adversarial red teaming only): No notification, SOC will treat all traffic as hostile and act accordingly with blocks, mitigations, active defenses, etc.

Access Control Lists (ACLs)

Are there any ACLs that our IP needs to be added to in order to ensure testing thoroughness?

It is advised, but not required, to add our testing IP(s) to any special access ACLs that may be in place for certain services. Although risks related to discovered issues with these services is lower due to the ACL(s), there are certain scenarios in which they would still be vulnerable: firewalls can “fail open” and permit traffic, the authorized IP sources could experience compromise and be used to attack this server, the authorized IP sources could “go rogue”, firewall rules could inadvertently be changed and expose this service, etc.

White Box Items

General

Although a white box approach is always recommended, it is not required. However, if a white or gray box approach was requested/scoped, there will be certain items that are required without re-quoting.

As discussed during scoping, black box testing is over 95% less thorough, can cost 2-3x as much, and results in a much lower ROI. So even if a black box has been quoted, the client is still encouraged to provide as much information in this section as they are comfortable with.

Reported Limitations: If certain items in this section are not provided, specific tests may not be able to be performed, or performed as thoroughly. For that reason, anything not provided will be listed as a testing limitation in the final report.

White/black box balance

Common Options (from common to least common):

• White box – anything requested, prior to testing
  • By far the most common testing that is performed and is often required if you are using this as part of a compliance requirement.
• Light gray box – all requests allowed by policy to be fulfilled
  • Second most common testing approach for organizations with dated policies.
• Gray box – Anything that could reasonably be obtained, given enough time, will be given upon request during testing
  • The “darkest” we ever recommend going. ROI starts to plummet here.
• Black box – spend the bulk of time performing recon, not testing
  • Our sales department and testers love this approach, but it is never advised for clients due to extremely low ROI.

Architecture Diagrams / Network Map

Anything you have, even just a rough sketch or narrative walk-through, can help with testing.

Endpoints

List of all of your endpoints (FQDNs, hostnames, IPs) so we can do a diff after host discovery to ensure we’re not missing anything important. This information has commonly helped to identify ACL issues early on in testing, which would have otherwise led to coverage gaps. Additionally, although we perform host discovery of all three RFC1918 ranges during internal testing, having a list of CIDRs or DHCP export with known live hosts allows us to confirm our testing platform is able to reach the intended scope.

Endpoints – External DNS Records

Export and provide all external DNS records from all domains you control (both in-scope and out-of-scope entries…we will filter the list accordingly). A, AAAA, TXT, CNAME, etc. records.

Endpoints – Internal DHCP/DNS

Provide internal DHCP and/or DNS records. This helps us confirm our discovery processes are seeing your known assets. Alternatively: Provide a list of known internal CIDRs (below)

Endpoints – Known internal CIDRs

List of known internal CIDRs being used. Note: We can generate the list of CIDRs from the DHCP export above (if needed).

Internal VDI/workstation

Preferably, this should be a “gold load” configuration that a typical employee has on their workstation.

This allows us to monitor for things such as account lockouts during various enumeration testing that will be performed, as well as provides the white box access for testing – user lists, email lists, password audit, etc.

Password & Acct lockout policy

Provide:

• minimum requirements for password length, complexity, number of passwords remembered, maximum password age (if any), etc.
• lockout details, to include lockout threshold, length of time/window, lockout duration, etc.

Examples:

• Windows/Active Directory:
  • Note: Typically, can take a single screenshot of both
  • Default Domain Policy -> Computer configuration-> Policies-> Windows Settings->Security Settings -> Account Policies -> Password Policy
  • Default Domain Policy -> Computer Configuration -> Policies -> Windows Settings -> Security Settings -> Account Policies -> Account Lockout Policy
• IAM solution policy – ex. Enzoic
• Custom authentication provider
• Other services – ssh, ftp, mssql, postgres, mysql, telnet, http(s), etc.

Note: Unless otherwise directed, we may perform default credential checks against some or all of the above services.

Source Code

Read-only access to code for all custom web apps and mobile apps – zip export, read-only access to git repo, etc. Having access to source code is extremely valuable in our ability to detect flaws in custom code.

For now, please share with the following, but once tester(s) are assigned, we may need to add additional people: robert.neel@penconsultants.com / redeemedHacker

Log access

Having access to backend logs associated with the architecture or web application greatly increases our thoroughness in identifying input injection vulnerabilities that may otherwise avoid detection from a black box testing perspective. Given that input injection vulnerabilities are a leading cause of data breaches, we would highly recommend this access.

Backend Server Access

Especially for web app testing. Ability to see processes, network traffic, file systems, etc., usually through SSH console access.

SQL Server/DB Access

Especially for web app testing. Read-only access to all relevant databases and tables.

Tenable.io access (if applicable)

Assuming you want us to merge in your tenable.io scan data with ours and create a single report, please provide access or upload a compressed scan export during testing.

Credentials – General

Why provide credentials? Why does PEN Consultants need them?

• Most engagements in the modern age are scoped as white box, which requires testing credentials for authenticated scans and testing. Changes to this approach require re-scoping.
• There are numerous tests and checks that cannot be run without various levels of authentication, thus leaving gaps in testing coverage and likely missing vulnerabilities in your systems.
• Testing will be at least 95% less thorough without credentials alone. Ex. Not much past the login prompt can be tested without credentials (which an attacker is likely to obtain). ROI is higher when local admin/root access is given to all in-scope systems, as testing thoroughness is increased twentyfold (20x).
• Many compliance and testing standards require authentication.
• An industry-recommended white box scan will require testing credentials for authenticated scans and testing – Network scans, host-based scans, web service/app scans, etc.
• More about white box testing can be found here: https://penconsultants.com/whiteboxApproach

Notes:

• Local/Appliance Accounts: For endpoints/services that are not using your central authentication provider (ex., Active Directory), you may need to create additional, local accounts on those systems.
• IAM solutions (ex. CyberArk)
  • We need actual credentials for the “scanning” account(s) with local admin on all in-scope endpoints – those creds are loaded into the various scanners we use during the initial automated portions of testing. Specifically, they need the ability to authenticate/communicate with non-interactive protocols (RPC, WMI, SMB, etc.).
  • Those credentials, when checked out, must be valid for at least 10 days, preferably 30 days. Ex. To support longer-running processes.
  • Note: Using an IAM proxy to RDP to DCs (for example) for interactive access is fine, as long as the credentials give access to the above-mentioned non-interactive protocols as well.
• Naming convention: We have no preference, but would recommend something descriptive so you remember what it is, not just during testing, but 9 months down the road. Examples for the standard user and domain admin:
  • pentesters_user
  • pentesters_da

MFA / multi-user access

For admin account(s): MFA can NOT be enabled on the account(s) we scan with (the highest privileged account) as it would not be possible to process the MFA. To mitigate this risk, this account could be restricted from external access – we’ll use the lesser privileged account(s) for external testing.

For standard user accounts: There are multiple testers on the engagement (typically). Because of that, the test credentials will be used by multiple testers, which complicates access that requires MFA. A few options to consider:

• Option A – Team MFA:
  • Create one account, set up SMS or PIN-based MFA with the information provided in the kickoff document
  • We can share the creds and distribute the OTP on our end, as needed.
  • OR: if MFA allows multiple phones to be assigned, we can also share the creds internally among testers.
• Option B – Individuals:
  • Set up unique accounts for each tester (w/MFA). This could become burdensome, but ensures attribution and allows for individual MFA. Additionally, we would still need a “scanning account” without MFA.
  • If this option is chosen, please set up the initial accounts with the details provided in the kickoff document, and once testers are assigned, we’ll provide the additional user contacts.

How to send credentials to us 

• Notes:
  • We change passwords once received, to further minimize risks
  • Upload a .txt or .docx document to Box (under the “From Client” folder) with URLs, access instructions, usernames, etc., or even just place them within this document.
• For the password(s) themselves, a few suggested options:
  • Send them via Text (or Signal) to the cell number listed in the kickoff document next to “MFA” which should be just a few rows above this line.
  • Upload them in a password-protected zip or 7z to Box, then text/signal the archive password
  • Use your own “secure mail” solution or equivalent

Credentials – Standard User Account

Example: What you would give a typical non-IT employee, user, customer, etc. Alternatively: In some cases, this account may be self-provisioned.

Although scanning and testing are performed with the admin/root credentials, identified issues are confirmed with user-level credentials, which helps to prioritize the findings.

Email Account

Corporate email account provisioned and tied to the above standard user account. 

Credentials – Domain Admin Account

Several tests require (or operate more fully with) a domain admin account.

• External/all: user password audit, password spray, MFA adoption, and some bypass testing (ex. pushes to users), certain pivot tests. etc.
• Internal (if applicable): Kerberoasting / Silver|Golden Ticket type testing, AD/forest trust issues, some AD/GP policies checks, detailed testing of permissive network shared and sensitive data storage, etc.

Credentials – Local Admin Account

Example: What you would give an admin, senior IT employee, developer, etc.

Admin/root/SSH/RDP – on all workstations, servers, external services, etc. If possible, the preference is to have an AD account (for example) which has local administrative privileges on all endpoints, but NOT domain admin for this role. Alternatively, a Domain Admin account could be used in place of this.

Other Accounts and Services (as applicable)

• VPN – if not already provisioned with the standard user account
• SSH – Often, these credentials have to be created locally.
• Anything publicly accessible
• Administrative (read-only) access to all cloud resources (will give us the ability to run a high-level cloud audit as well)
• Administrative (read-only) access to any 3rd party services that are integrated with the in-scope systems – ex. payment processing, SMS/email services, MFA solutions, etc. This will give us the ability to track back-end errors and identify potential implementation weaknesses.
• Any others we may have missed

Dropbox – Our Internal Testing VM:

For internal networking testing only – download and import our VMDK/OVF image that will be used to conduct the internal testing.

• Whitelist Anti-Virus/malware
  • If not already, ensure your AV solution has exclusions for VMs (at least ours)
  • Example: https://docs.microsoft.com/en-US/troubleshoot/windows-server/virtualization/antivirus-exclusions-for-hyper-v-hosts
  • Note: Our VM contains many attacker tools that AV will alert on, making it a noise problem at minimum, and potentially cause you to have to reload it.
• Download the archive file from our website.
  • See the hyperlink provided in the kickoff document.
• Confirm checksum, if you are able
• Unarchive the zip file
• Import VM:
  • NOTE: These are very abbreviated instructions, assuming you’re already familiar with the process. Please ask if you need additional guidance.
  • VMware vSphere / ESXi: Right-click the host → Deploy OVF Template.
  • VMware Workstation / VirtualBox: File → Open / Import, then follow the wizard.
  • Microsoft Hyper-V: Convert .vmdk to .vhd with qemu-img, then create a new VM and attach the disk.
• Before booting, confirm or change to the following specifications:
  • 100 GB disk space preferred, 50 GB minimum
  • 8 cores preferred, 4 cores minimum
  • 16 GB of RAM preferred, 8 GB minimum
  • NIC speed: 1gb+ preferred, at least 100mb minimum
  • Bridged, not NAT, mode (or equivalent) to ensure network ports in VM are accessible from other endpoints
• IP address
  • Option A: We have configured this with DHCP, with the assumption you’ll be setting the IP via a DHCP lease or within the virtualization dashboard.
  • Option B: If needed, we can provide you with login credentials and a script to set a static IP.
• Ensure VM can:
  • connect outbound, over tcp-22, to the listening post provided in the kickoff
  • touch all internal endpoints that are in-scope – I.e. no internal firewalls or ACLs blocking access
  • make web requests (over http & https) to the following domains:
    • ubuntu.com
    • archive.ubuntu.com
    • security.ubuntu.com
    • kali.org
    • http.kali.org
    • tenable.com
    • www.tenable.com
    • plugins.nessus.org
    • downloads.nessus.org
    • plugins-customers.nessus.org
    • plugins-us.nessus.org
    • cloud.tenable.com
    • appliance.cloud.tenable.com
    • tenablesecurity.com
• Additional notes (if needed)
  • SSH interception/inspection and DPI must be disabled for the destination above, or SSH may not work, and our VM may not be able to connect out.
  • Integrating an IAM solution (ex. CyberArk) for that would not be practical (we have a bit of experience trying) due to two issues – (1) the tunnel back to our needed infrastructure would not work properly, (2) the profile encryption we have set up on that VM does not play well with it.
  • For clarity, our VM reaches back out to us over SSH to establish a secure tunnel back to infrastructure we have set up for this engagement.
  • Our external server firewall is temporarily allowing all inbound tcp-22 connections until the VM calls back, and then will be restricted to authorized IPs only.

Demo (ex., for web and mobile app testing)

Please provide a few days/times that you are available to give us a demo of the application – Walk through how a user navigates/operates, an admin, highlight areas in need of extra focus, discuss architecture, confirm access & permissions, etc.

Data Integrity

Disaster Recovery

Confirm you have total disaster recovery in place, such as off-site backups of data, offline backups of websites, etc.

If this is Network Testing, we don’t anticipate any issues with data corruption or loss, but it’s always advisable to ensure your backups are up-to-date and verified before testing starts.

Environment Risk Tiers

What level of precaution is required during testing? Note: These coverage descriptions reflect restrictions imposed by the environment tier, not the overall completeness of testing. Actual coverage also depends on the engagement tier (Basic, Standard, Premium, etc.) selected.

• Tier 1 – Disposable (MOST COMMON FOR WEB/MOBILE TESTING IN A CLONED/TEST ENVIRONMENT)
  • Environment: Non-production; easily rebuilt or restored.
  • Impact: Minimal; temporary disruptions acceptable.
  • Approach: Aggressive testing allowed, no restrictions.
  • Coverage: No meaningful limits on what can be tested.
• Tier 2 – Recoverable
  • Environment: Non-production; recovery requires time/effort if issues occur.
  • Impact: Possible delays for teams but no business impact.
  • Approach: Use “safe” methods early; more aggressive testing at the end.
  • Coverage: Nearly everything can be tested, but small stability concerns may require holding back a bit.
• Tier 3 – Redundant
  • Environment: Production; isolated or redundant systems that can be restored after testing.
  • Impact: Limited; some disruption possible but recoverable.
  • Approach: Increased testing speeds acceptable; monitor for service impact.
  • Coverage: Most areas can be tested, though some techniques are excluded to avoid production disruption.
• Tier 4 – Business-Tied (MOST COMMON FOR CORPORATE NETWORK TESTING)
  • Environment: Production; redundant systems, but linked to real business use.
  • Impact: Potential user/service disruption.
  • Approach: Restrict to standard, slow scanning, safe checks, off-peak windows, monitoring.
  • Coverage: Several attack paths must be skipped; some blind spots expected.
• Tier 5 – High-Impact
  • Environment: Production; core business systems or sensitive data at risk.
  • Impact: Significant business impact or data loss possible.
  • Approach: Greatly limit testing tools; proceed slowly and carefully with constant monitoring.
  • Coverage: Many areas skipped or simulated; notable blind spots.
• Tier 6 – Safety-Critical
  • Environment: Production; mission-critical systems where failure could endanger lives.
  • Impact: Extreme – loss of life, safety risk, or regulatory catastrophe.
  • Approach: Manual testing only; two-person verification required; prefer mock/test environments.
  • Coverage: Only very limited tests possible; substantial blind spots.

App Services Approach

If testing is for Web or Mobile App Services, there are two common approaches:

• Option 1 – Replicated/Test Environment (test data purged after testing): Stand up a production-like test environment we can thoroughly test (“go to town”) and later tear down. The key is that it closely mirrors production, including third-party integrations, and all test/junk data is purged at the end.
• Option 2 – Production Environment (test data visible to active users): Test directly against production, but we significantly scale back the depth and aggressiveness of testing to avoid disruptions and avoid injecting junk data (e.g., thousands of test records). This typically means running about half the number of tests; those we do run take longer and are scaled back significantly to avoid data corruption or junk-data pollution, so the overall testing hours remain the same, but with almost certainly fewer findings.

The main concern with testing web/mobile services:

• Production Environment: potential for both disruption (e.g., consuming bandwidth) and destruction (e.g., injecting junk data or corrupting existing data).
• Cloned Environment: potential for certain features not being implemented or not working (ex., first and third-party integrations) and missing vulnerabilities because of it (e.g., if the features are not available, they cannot be tested).

For context, our clients choose Option 1 in about 25-35% of engagements, and Option 2 for about two-thirds.