Death by Exception
November 15, 2011
As an assessor, I’m tasked with reviewing client’s policies to not only to determine if they are sufficient, but to also evaluate if they are being adhered to. For years, if a policy or procedure was not being followed, I’d assure my clients that exceptions were OK and absolutely expected just as long as they were aware of them. The advice was well received and clients put procedures in place where exceptions were identified, reviewed, and approved…and everyone lived happily ever after, THE END. Naïve, I know.
And so these organizations continued with their exception procedures, year after year, until eventually exceptions became the rule. Month after month, review after review, exceptions became a strategy for departments when they needed to bypass certain security requirements, thus creating a black hole of exceptions never to be seen or heard from again. Eventually, meeting the requirements became the rarity.
Exceptions, much like compensating controls in PCI, were not meant to be permanent. They are usually given when certain processes, practices, applications, or implementations are unable to meet established requirements. The idea is to get the exception so operations can continue, not to get it and forget it. Although organizations have become great at obtaining exceptions, they’ve failed at managing those exceptions. As I am exposed to new and different environments, I’m quite surprised at how common exception black holes have become.
As I mentioned, exceptions are not permanent and should be reevaluated periodically. When an exception is originally requested, it should also include remediation plans for eventually complying with the business requirement. Their periodic review should be an evaluation of the status of the remediation and to determine if other actions need to be taken to mitigate risk.
What I’ve found is by not managing exceptions; organizations have put themselves in a precarious position. Although exceptions are usually reviewed and approved by a central committee/group, the organization doesn’t do a good job of tracking the amount of exceptions they have approved. By not tracking exceptions executives lose perspective on the amount of risk they have actually accepted. After performing an assessment on a few hundred departmental applications at one company, management was shocked to discover that almost 90% of the applications did not comply with policy but instead went through the exception process. This revelation is what most organizations are missing out on. The “exception big picture” is hidden from upper management while maintained in departmental vacuums. Exceptions are not a get out of jail free pass, but are last resorts that still need to be reviewed and managed.
C.Y.A…The road to nowhere
August 13, 2010
Motivation…..Many things act as motivators for how one does their job. Job satisfaction, meeting contractual requirements, or achieving compensation goals are a few reasons why people do their jobs to the best of their ability. Over the past 3 weeks I’ve had conversations where the term C.Y.A. (Cover Your A$$) has been uttered. I have heard this from both clients and from within my own organization which when considered in their context created a light bulb moment for me both as a security assessor working with my clients and as an employee working for an organization that offers security assessment services. When CYA is a critical part of doing your job, the amount of overhead one creates for oneself can be disruptive.
During my career doing both infosec and PCI security assessments, I’ve seen many examples of CYA. Usually when interviewing business units I tend to discover that credit card numbers are being written down, emailed internally, and then stored indefinitely. This typically is both a surprise to my point of contact as it is to me. When asked why, the response is always a whispered, “CYA”. For reasons that may be obvious to other assessors, this response has the hairs on the back of my neck standing. After further discussions, further paper trails, and scope creep, I have realized that CYA is no longer an admirable quality, if it ever was one. Because several business units within organizations tend to follow a CYA course of action, the amount of “sensitive” information that is being shared internally as well as being sent off site expands the scope of the assessment dramatically. And with increased scope comes increased effort to control and bring into compliance these various business units.
Now one might write this off as an organizational and procedural problem, but when I hear the term CYA explained to me internally when dealing with clients, again I was struck with a foreboding feeling. In the name of CYA, the amount of time I am required to spend on each client and corresponding paperwork seems absurd. True I am billable for that extra time, but I now have to fit this into a schedule that was already packed with my “regular” duties.
As an assessor CYA is showing itself to make it increasingly more difficult for organizations to adequately protect sensitive information. At the same time, internal CYA is making assessors more skittish in not only how they handle their clients but in what they are requiring of their clients during assessments. CYA has even recently entered my realm of thinking and has spooked me. Spooked me because of the recent unpleasantness I’ve but some clients through…No, I am not perfect…shocking!! With that said, I’m not naïve in thinking CYA is not in the back of everyone’s mind when doing their job, but CYA should never be the driving factor in the day to day operations for either me or my clients.
What difference does it make?
July 26, 2010
As a major part of becoming and maintaining PCI compliance, organizations are tasked with performing routine/periodic tasks over the course of a year. And during their annual PCI re-validation, the organization is required to provide evidence that these routine tasks have been performed. But what happens when the QSA determines that the organization did not keep up with their homework? How does it affect their PCI compliance?
Short answer….it doesn’t.
Merchants and service providers (SP) are required to perform the following routine tasks which a QSA then needs to validate as having been completed:
- Daily review of audit logs
- Quarterly internal vulnerability scans
- Quarterly external vulnerability scans
- Quarterly scans w/wireless analyzer
- 6 month review of firewall/router rule sets
- Annual risk assessment
- Annual review/update of infosec policies
- Annual internal penetration test
- Annual external penetration test
- Annual security awareness
- Annual incident response test
Of course if this is an organization’s first attempt at compliance, they will be unable to provide a “year’s” worth of evidence that the above has been completed. But, if this is not their first time around the PCI block, then the organization should be able to provide a year’s worth of evidence. However, what happens when a merchant or SP missed a quarterly scan or it’s been over a year and they have yet to conduct IR testing? One might think that they’ve missed their mark to be compliant for that year, and another year needs to go by. But that is not the case. If an organization failed to meet a task, they are simply asked to promise (and mean it this time) in a strongly worded policy/procedure to have it done in the required timeframe next year. As for the current assessment, the organization just needs to get it done and submitted to the QSA for review, even though it is outside of the task window. As long as it’s done sufficiently, the QSA has no choice but to mark it “In Place.”
What’s the point? I’d like to see a QSA that actually tells the organization, “Nope, you missed the annual window. You’ll have another chance to be PCI compliant next year.” I know I am being harsh but where are the consequences for not taking these tasks seriously knowing they can just get it done after the fact? Does a breach have to occur in which the card brands, jumping up and down, pointing their finger proclaiming that “at the time of the breach, the organization was not PCI compliant.” By then it would be too late. I think most of us can agree that the tasks identified are part of a comprehensive infosec security plan; however, I consider this is just another example as to why the PCI standard itself needs to be re-evaluated on how it is being applied.
Letter to the client
July 7, 2010
Dear Client,
I know you have reservations about going through an assessment so I propose a mutual commitment of things we can both do to make this process as painless as possible:
You agree to do the following:
- Gather as many of the policies, procedures, and supporting documentation available before I arrive. In other words, if you say you do something, be able to provide some sort of evidence that it is being done.
- Schedule interviews with the people who “do” the actual work, not the managers that think they know what’s happening down in the trenches. I’m sure you think you know what is supposed to occur, but in the interest of not wasting our time, I’d prefer to speak with those that actually do the work.
- Identify your environment. Make sure there is an updated network diagram and a matrix detailing all of the systems that deal with sensitive information. For example, during the assessment is not the time to discover the backup of the production database under the receptionist’s desk, which was replicated “just in case.”
I agree to do the following:
- I will not be your enemy. My goal is not to publicly humiliate you or your staff. The goal is to work together to identify any gaps in your infosec program in order to make it better.
- I will not know everything there is to know. I encourage you to “push back” if you do not agree with anything I say or find. I welcome conversation as it better helps me understand your environment.
- I will not ask “trick questions.” If you want to mislead me, then that is on you. I am going to ask you what I need and want to know.
An assessment can be a learning experience and by agreeing to the terms listed above you can expect the same courtesy in return. As this can help in easing the pain that is….A security assessment.
Disclaimer: I do not speak for all infosec assessors, neither good nor bad. These are my own thoughts based on my own experiences.
Fear Not the Assessor 