An open letter to firms requesting security assessments (especially IT consultants selling “security assessments”):
Running a scan is not a security assessment. During discussions with prospective clients, we often hear a sentence that makes true cybersecurity professionals cringe: “but the other company is much cheaper than you, and they have a scanner”. While we too use discovery and vulnerability scanners, the engagement fee for a security assessment is not determined by our use of these technologies. We use them because they are a valuable tool in the toolbox, but an assessment does not end after the tools has been run. We often rebuttal, “Other countries will happily scan you for free.” Many people and countries are already scanning and reporting on your network. The difference is that it might not be you requesting the report.
When a reputable IT security firm performs a security assessment, an IT General Controls Review (ITGC), a vulnerability assessment (VA), or a penetration test (PT), etc., the results of the assessment must be evaluated to classify the risk to your organization. The vulnerabilities identified must be evaluated against the target within which the vulnerability was identified. No tool or ‘blinky-box’ with pretty orange or yellow lights is a substitute for interviewing the data owners to identify where the true risk lies. When our teams of IT Auditors or IT Security assessors begin an engagement, we spend much of our time talking to key organization stakeholders to understand what and where important data is stored. We want to harness a baseline understanding of what the “good” is, so we can acknowledge and evaluate what the “bad” is.
A key, unique element of the way that we perform our assessments is our personal review of vulnerability scanning results. The impact to your organization of identified vulnerabilities is not created equal. An informational “INFO” rated finding (which is the lowest possible rating) may be CRITICAL if the context was identified on an internet facing connection. At one point, our firm was issued a request for proposal for which we put together a response. One competing firm’s bid was roughly 10x cheaper than the next highest (not us). I was so surprised, that I asked the presenter how they could afford to offer such low pricing. Per their team member, they “had a tool” that performed an evaluation of and reported on all 20 SANS Critical Security Controls. This claim was outlandish. No tool on its own could interrogate the inventory of hardware and software, identify security configurations, test the organization’s backups, verify the encryption strength of a wireless connection, interview staff about the existence of an incident management response plan, and perform the other key elements of an assessment.
The next time a security assessment proposal comes across your desk, ask the firm how will they quantify the results of the scanner? Will you receive the raw output, or a pretty HTML or PDF with 100 pages of critical items that need to be addressed right away, without any context to your business operations? Ask how much time is spent learning about the organization, instead of just “running a scan”. Who knows, picking the right security vendor may even save your business.
Sincerely,
Michael Hammond
Director, IT Audit & Security
