Hope you all had a great weekend.  I had meant to point you earlier to a FAIR analysis that Chris Hayes did over at his Blog.  But I’ve been a little busy, and before I could mention it, Stuart King put up a kind of angry response on his ComputerWorld blog.  Snark aside, there are a couple of other really troubling aspects of Stuart’s reaction to Chris’ analysis that I thought we could talk about this morning.

The problem is that (Chris’ analysis is) completely impractical. I’ll take a recent, and fairly typical situation as an example. I was taking issue with the manner in which remote access was being provisioned for a third party vendor to connect to a system hosted by one of our European business units. To cut a long story short, it was not only a breach of policy but highly insecure. I wanted the access to be disconnected, the business unit director wanted my risk assessment. And he didn’t want to wait for it.

To quote Chris Hayes, spending time on working out the expected effectiveness of controls, over a given timeframe, as measured against a baseline level of force was not going to pacify an angry Italian fearful that my decision was going to cost him money. He wanted my explanation of the risk and more importantly, what I was going to offer as a solution to keep his business functioning

As Chris is someone who actually does this for a living in a large company, and this is typical of his actual day job, I really find Stuart’s “impractical” comment to be, um, misinformed.

Also, I think Stuart mistakes the purpose of a risk analysis.  The purpose of the risk analysis is not to force someone to be “secure”, but to provide knowledge for decision making.  Using it as a “hammer” to knock in the nail of your personal risk tolerance impairs efficiency and in the long run retards “security” as it creates political resentment.  Seriously, who cares if something might violate policy or not in a pre-implementation discussion?  Policies are not stone tablets handed down from on high, they are state-in-time codification of the data owners risk tolerance.  This risk tolerance changes sometimes, and that’s OK.

To that extent, I appreciate (and I’m sure Chris does, as well) that risk analysis does not create rationality in the data owner.  Someone who sees you as a speedbump on the route to progress they may not be ready to appreciate your point of view even if it is stated in the most rational manner possible.   But it’s worth noting (and Stuart’s example is indicative of this point) that risk analysis does not create rationality in the analyst, either.  If one is being so “security minded” as to ignore the risk tolerance of the business owner - we’re bound to get a reaction similar to that Stuart encountered.  In fact, a practical risk analysis like Chris performed on his blog, done in 30 minutes, should identify the critical point of disagreement between Stuart and the data owner (again, Stuart doesn’t own the data, the agitated Italian does).

But let’s stay rational and open to alternatives to what Chris offers.  Stuart states his approach to risk analysis as:

When I need to document a risk assessment I use a very simple form: list the threats, state the level of vulnerability, list the associated operational costs and potential revenue hits. High, medium, or low risk? Describe the controls and options. Write up who needs to do what, and how much of their time it’s going to take. Job done.

At first glance, I don’t think what Chris has done is any less efficient, and it provides greater insight (using Frequency and Capability instead of just ‘listing the threats’).  But what is key here is that Chris’ approach is consistent and defensible.  Less generous risk geeks and CSO’s I know would have no little difficulty with Stuart’s approach.  But to particularly answer Stuart’s main objection (impracticality) I would offer that with practice, Chris’ work is probably quicker and easier than Stuart’s described process as it eliminates much of the ambiguity an immature risk analysis creates - reducing the need for further discussion and arguments with data owners (regardless of disposition or nationality).

Finally the irony of Stuart’s post is that the reason he had this confrontation may in fact be because he was incapable of bringing a salient model for risk to the table, one that identified the factors that create risk and developed a defensible belief statement concerning risk.   We’ll never know if one would have helped him in this isolated instance, but I can tell you that in organizations like Chris’, good risk models and strong risk anlayses create operational efficiencies, reduce costs, and streamlines intra-departmental communications.

Pre-Script - It should be noted that the outcome of this discussion - in the last paragraph - is one smart way you can approach the “We need to reduce your budget” discussion (if that discussion hasn’t come already).

I’ve often read people who say that we (security, risk management) need to “think like the attacker”.  And when you read this sort of article, that usually alludes to trying to anticipate the tactics an attacker might use to mess with your C, I, or A.  Smart stuff, that, and very useful when architecting security solutions.  But as I was training some folks Monday, I was thinking in the back of my head about Threat Capability (TCap) in FAIR.  As you might know, we like to estimate the capability of a threat to apply some level of “force” against our assets.  This ability to apply force is a byproduct of the attacker’s skills and resources.  And thinking of how an attacker applies skills and resources, I came across another way we might “think” like an attacker.

Traditionally, I’ve thought of “skills” as being a byproduct of the toolset an attacker has.  This mindset probably stems from my time with Penetration Testing teams, where in the process of scoping the  PenTest I would ask our clients to select the level of effort that they wanted us to throw at them.  If a client chose “high” we’d throw every ‘spoit we had at them.  If they chose “low” we’d limit ourselves to a more commonly available toolset.

But while the resources part of TCap is time & materials (money) - the skills are really more than just the toolset.  Skills would include the ability of the attacker to be creative and innovative.    As an example of that innovation from those PenTesting days - when we got a “high” effort request, we would always try to couple that with some “social engineering”-type of attack, or some unique means of delivering an existing exploit.  Our creativity was not necessarily a byproduct of a unique exploit or tool we had, but the process by which we might deliver pre-existing or commonly available exploits.  I remember when we first got ahold of a handful of 32mb thumb drives (hey, 32mb was huge back then) and “dropped” a few in the lobby of a client’s retail space.  The keystroke loggers and phone-home script weren’t new, but using the thumb drive as delivery vehicle certainly was.

So I’ve started to really think about this concept of innovation, and how if “thinking like an attacker” means to be innovative, we ought to do the same.  I’ve been thinking of two main categories of innovation this morning.

INNOVATION

The first I’ll call Technology Innovation.  And by Technology Innovation, I mean some new, unique, “ahead of the curve” technology that an attacker can use against us.  The obvious example of which is a zero-day.  It’s that “high” tool set our PenTesters would use against the clients.  For security departments, this might be the latest security product designed to enhance our ability to P, D, and/or R.

Alternately, we can be creative in the way we deliver (manage) existing technology.  I think of this as Process Innovation.  It’s doing more with what we already have, just like the PenTest team would be creative in the delivery of an existing exploit.

Unfortunately for us - attackers have traditionally had quite a leg up on us in terms of Process Innovation.  It is much easier fro them to be creative, as they are free of political constraints and bureaucracy.  In contrast, when the security industry tries Process Innovation, the results are checklists and “standards”.  It’s committees and consensus.  An extreme example of which might be something like SABSA - a great work if you want to understand some very smart people’s comprehensive understanding of organizational security  - but the “adoption”of which will do very little to help you be innovative in P/D/R.

It’s worth noting that ultimately, this is one reason I don’t like regulatory compliance efforts - they simply serve to prove how mundane your security department is,  wasting valuable resources that could be spent on creating ways to be more effective.

PROCESS INNOVATION AS A SUBSTITUTE FOR TECHNOLOGY INNOVATION

As we come to the close of 2008, some surveys suggest that security spending isn’t horribly impacted yet by the economy (the latest from E&Y points to only 5% of their respondents getting budget cuts).  But if this is a protracted downturn, and because InfoSec is an operational expense, I would expect cash to become more and more difficult to keep.  And regardless if technology spends do slow, I believe it makes sense to think about Process Innovation because I see Process Innovation as a means to increase effectiveness without significant capital expenditures (effectiveness increases because our ability to manage risk has a direct correlation to the amount of risk we have).

The bad news is, of course, that great innovation is hard.  It is R & D.  Failure is usually a pre-requisite to success.

The good news is, our current state is so bad that many of us don’t need to come up with a whizbang new way of reducing software defects in the SDLC as innovation.  Simply inserting a risk analyst into the PMO’s processes might count as a big enough victory. Be cautioned, though,  that if we’re substituting the risk reductions provided by technology acquisition - Process Innovation might actually be even more “expensive” as it requires us to expend political capital.   But there are (forgive the term) innovative ways to spend this political capital.

For example, by taking a second now and figuring out the 3 things that the rest of the organization can do to make your life easier, when that “I need to reduce your budget” talk comes, you can be prepared to negotiate.  Get a political capital “loan” or “investment” from the C-Suite reducing your budget.  Something to the effect of: “I expected this, and am happy to give up my budget.  But if our tolerance for risk hasn’t changed, what I’d like to do is get you to personally back my office on three projects I’ve identified that can reduce our risk without requiring significant capital expenditure.”

There’s been quite a few people talking about what sorts of strategies make sense for security and security departments in a downturn.  And they’re all very good - but there’s one thing that I’d like to add.

One easy, inexpensive way to actually increase your effectiveness in 2009 is to, right now, make a quick review your risk management processes.  As you take a look at how you’re using risk in your organization, I’d ask you to make sure that those processes are providing value for the energy you’re spending.  If they’re not - if you’re not successfully using risk within security and with the other lines of business that you serve - then I’d like to invite you to  come take advantage of RMI’s public training session for 2008, held in Columbus, Ohio on December 10-12.  >A brochure is here<.

For three days and $1,995 - you’ll get real answers to many of the commonly voiced frustrations RMI hears concerning risk & risk management.  Answers around measurement, application, communicating risk to other lines of business, heck, basic answers as to what risk is and how to get consistent, defensible values that actually mean something.

Not to mention - Strengthening your Risk Management processes increases your ability to manage risk, which reduces the amount of risk you actually face.

NEW TO THE PUBLIC STUFF!

I’m personally excited because this is the first time that our public training will feature measurement “calibration” exercises and include excel tools to take home and use for quantitative FAIR analysis.  These are benefits we’ve only previously reserved for private client workshops.

I know that FAIR can help you and your organization, but as the sales guys always say, “don’t take my word for it”.  Here’s something we recently received (unsolicited) from the CSO of one of the 10 largest banks in the US, who has had several of his analysts receive this same basic training:

I would like to also add my deep appreciation for what FAIR and RMI has brought to (us) and how we go about the business of risk analysis. We have had some great conversations around risk with the lines of business that have ended very favorably for us.

More information can be found on RMI’s website here:  http://www.riskmanagementinsight.com/12_2008_training.html

Thanks.

Oh and tomorrow, we’ll talk a little bit about quantitative and qualitative risk.

==================================

UPDATE:  @MYRCURIAL from the great site Liquidmatrix says that I need to post the following warning:

YOU MAY NOT WANT TO PROCESS THIS PRIOR TO YOUR 11TH CUP OF COFFEE

==================================

Carrying on from yesterday’s post a bit, I’m happy to admit that Chris’ poem is right: we don’t have nearly the information we need now when we’re supposed to have “control” over our assets, putting things in a hosted/asp/cloud/buzzword model ain’t going to help our quest for visibility. My intention was/is to show that you need visibility (in part one) and then today explain that unfortunately, that’s only half the picture.

Today’s follow-on is about the fact that whatever visibility we can contractually enforce (be it in the “cloud” or in our own perimeter) has to be informative (Amrit, this is why I was plugging you with those variance questions on Twitter yesterday).  That is, we can ask whatever IT department (ours, theirs, whomever) for all sorts of information, and maybe they’ll even give it to us.  But we’re not really ready to:

• Know what to ask for
• Use it to create wisdom

A really salient example of this from outside IT hit my browser this morning.  Now it’s not at all my intention to be political or endorse one candidate over another.  Those who know me know I’m fiercely independent.  But this morning there’s a headline on a well-read news website about how one candidate is now “+2″ over another in a Gallup poll of “likely voters”. The source is here.

That is a screen grab from Gallup’s website that shows the “+2″.   I have to ask - how informative is this information?  Part of the problem is that Gallup’s methods are hidden as some sort of “secret sauce” (their FAQ section doesn’t help much, either).  But regardless of the quality of the measurement, this “+2″ has no context - we don’t really know what this information means with regards to an actual election.  Nor is there any predictive element (I hate the using the word predictive, but it’s common nomenclature - so there you go).  We don’t have what we need from this Gallup poll to create wisdom about the ability of either candidate to be elected.

Allow me show you what I mean by way of contrast.  Take a look at Nate Silver’s work at http://www.fivethirtyeight.com/.  Now I’ve been long familiar with Nate due to his work in baseball.  He’s been at these sorts of ‘predictive’ analytics around our shared passion: creating wisdom from baseball statistics.

What Nate is doing at 538 is applying that acumen from his baseball work to the political process.  He’s breaking down the vote not just on popularity among likely voters, but in the context of the electoral college, accounting for variance and uncertainty, running Monte Carlo simulations and taking into account all sorts of polling information.  The result is really quite amazing. Here’s just one graph he presents - it’s the most similar to the Gallup one above, but you should really visit the site to understand the difference in quality of information and to check out the predictive elements he creates.

NOT ALL INFORMATION IS CREATED EQUAL, AND NOT ALL  JUDGMENTS ARE CREATED EQUALLY

And take a look at the contrast, here:

On one hand you have Gallup giving us a “+2″ advantage to a particular candidate.  Now Gallup themselves draws no conclusion but, as digested, how many readers do you think take this as evidence that the election is *really* close?

On the other hand, 538’s predictions show a 348/189 electoral college split, and one candidate winning 96% of the time in simulated elections.  That doesn’t seem close at all!

RISK MANAGEMENT

It is these predictive elements that we need in order to make better strategy and decisions.  I’ve been talking in the past about risk management’s inability to link current state to systemic causes, and this “context” is what predictive analytics provide.  We might have all sorts of visibility into our environment, and measurement of various amounts of variability that visibility gives us. But unless we have context to create wisdom, it’s all just, as Chris says, “machinations”.  We have to move beyond “+2″.

So Cloud/Grid/Utility/ASP/TimeShare/Whatever you want to call it - security will have to clean up our own mess first before we can do a good job with or without a perimeter.  Once we can start moving beyond “+2″ statements, then we can know what sort of visibility we require into an ability to Prevent, Detect, and Respond.

Crazy, Man.

Lots being written about the Cloud, most of it quite dark and gloomy.  In fact I’m surprised, that Hoff hasn’t got a preso spooled up called “The Toxic Cloud” or something similarly ominous for his next speaking tour.
That said, the Economist does a great job distilling the issue into a simple statement -

Cloud computing is a trade-off between sovereignty and efficiency.

Let me ask you -  if you had to put your money on one of those horses, considering your average profit-preoccupied business, which would it be?  I’d put my bottom dollar on the thoroughbred named “Cost Center Reduction”, to place.

WHO ARE WE TO STAND IN THE WAY OF “PROGRESS”?

I’m always fond of Jack’s rule that the role of information risk management boils down to three deceptively simple premises:

• Reduce Risk.
• Reduce Loss.
• Create Operational Efficiencies.

So it would seem antithetical to the charter of the Chief Security Officer to stand in the way of progress as embodied by “cloud computing” (not to mention dangerous to long-term job security).  And I think that this presents opportunities to discuss strategies for managing risk, strategies that aren’t too theoretical and have practical application (though actual “cloud” use by enterprises may be rare at this point).

ON RISK REDUCTION IN THE CLOUD (or, How To Learn From the Shortcomings of PCI DSS)

The good news is, there’s already a well-established model for managing the risk around outsourcing the processing of “confidential” information.  The bad news is, that model kinda sucks it.

The Payment Card Industry, known as the “PCI” or “meal ticket” to many in the industry, faced a similar problem with the introduction of GLBA.  As I see it (and I’m not at all close to the PCI, at all, so this is all just abstract soliloquy) the PCI had one of two choices when faced with the prospect of other people managing their sensitive information:

1. Accept the *massive* amount of GLBA risk their business creates and spend a TON of money to build out the infrastructure (both process and IT) to manage the consumer data themselves (in conjunction with the banks, of course) and never have it grace the computing systems of the retailer.  Or,
2. Transfer the GLBA risk down to the retailer and have them bear the majority of the risk (and cost of reducing risk to a level that might be tolerable to the US Government).

(Martin, you may recall our Twittering about PCI a while back.  This is the crux of my view on the subj.)

Now fortunately, the CSO’s of the world are going to be a little more “invested” in protecting the information they are stewards over, and unlike the PCI, will remain primarily responsible for the C, I, & A of the data in the Cloud.  The cool thing is, this actually presents a great opportunity to start building a meaningful model for co-management of risk!  In fact, we can take the PCI model of contractual risk transference but modify where it goes all wrong, and start working to create something better.  And we can start by euthanizing some faulty assumptions.

JUST HOW INFORMATIVE IS PCI DSS?

What might be the.greatest.mistake of the standards compliance mentality is the assumption of value for the past-state measurement.  That is, I believe that the CSO needs more than some “past-state” assurance in order to understand their risk.    If you look at the concept of “PCI compliance” it really is an examination of a past state of nature that is assumed to be relevant to current and future states.   Many people (myself included) are not at all convinced that this past-state is nearly as informative as those who mandate it’s measurement believe it to be.

That’s not to condemn past-state measurements as completely non-informative,  they most certainly are useful.  It’s just that no self-respecting CSO sleeps well because they were deemed “PCI compliant” 10 months ago.  They sleep well because they have good visibility into current-state information and confidence in their strategy concerning future-state (based on that visibility and the outcomes of sound IRM models).

MOVING PAST THE VULNERABILITY SCANNER INTO INTELLIGENCE AND WISDOM

So realizing this new importance (to me, at least) concerning visibility and IRM models, I’m lead to the conclusion that if we are to manage risk in the Cloud, we’ll have to move beyond “PCI Compliance” or the concept that some regular “audit” of controls in place at the host is all we need to understand our ability to manage risk.  No, the CSO must have good information concerning current and probable future states.   This is that “visibility” I spoke of above.  In fact, we’ll need significant amounts of piercing, transparent visibility.  And in order to gain that visibility, our insight into Cloud Risk Management must include significant provisions for understanding a joint ability to Prevent/Detect/Respond as well as provisions for managing the risk that one of the participants won’t provide that visibility or ability via SLA’s and penalties . These SLA’s must be expressed in measurable terms (more visibility), and those metrics must have their roots in the things that help understand how we manage risk (those aforementioned IRM models).

THE CLOUD COMPUTING SECURITY SILVER LINING (sorry couldn’t resist)

As I mentioned earlier, I do see an opportunity to create insight.  The need for visibility and IRM models would allow us to create a “guidance” if you’ll allow me to use the term.  Not a standard or a “best practice” to audit by, but simply a reference document that says “if you’re going to put information on somebody else’s systems and still hold some significant responsibility for that information, here’s the considerations, why they are considerations, and how you might go about collaborating on the management of risk”.

And I think that if we undertake this journey, there is going to be a lot of growth and risk management innovation along the way.  But keen insights into what it means to manage risk will be necessary, and secure and forthright collaboration will be of absolute importance.

I say that last bit because, if these pundits are right about the utility of a hosted computing model - the Cloud will happen regardless of the CSO’s ability or desire to manage it.

Sounds like the beginning of a joke, right?  So these two guys walk into a bar…

“The” Bruce Schneier and Marcus Ranum have an article up on TechTarget/Information Security Magazine called, creatively enough, “Bruce Schenier, Marcus Ranum debate risk management“.

Unfortunately, to get to the article, you’ll have to either already be a subscriber to IT Security, a subscriber to TechTarget, or go through the 20 minute process of signing up by giving TechTarget all sorts of “market information” about how you’re really Brandon Walsh, CSO of “The Peach Pit” Industries in Beverly Hills, CA 90210 (phone 714-867-5309).

For those of you who are already a TechTarget person, the link is above.  For those who aren’t, or those who just don’t have the time, I’ll summarize.  The “debate” is kind of awkward because both authors seem come to the same conclusion:

Risk Management, it’s something our profession should do, something humans do naturally, it’s necessary in business, but gosh - we don’t have enough data.

I’m not a cryptographer.  I don’t *nearly* have the insight on privacy and politics that Bruce has.  I’m not deep in IP communications.  I haven’t got a proven track record of innovation in IP Security products like Marcus has.  But here’s the thing, I hope you’ll never hear me pretend that I have the skill set to speak authoritatively on those subjects.  Heck, I wouldn’t claim to be a “risk” expert because I have a some insight into my shortcomings and what is needed to tackle such a complex problem.  But such a tepid article on something that (at least I think) is so important kind of, well, confuses me.

Why is it such a boring article?  I’m not sure.  Maybe because they’re just two guys who would rather debate the merits of specific controls or control activities (after all, their penetration testing debate was a huge success), but there’s no new information in the “debate”.  It’s the same old “insurance companies know risk because they have scads of data and we don’t have that” complaint. You know what?  I’m tired of hearing that line, so let’s talk about it.

HOW DO YOU KNOW WE DON’T HAVE THE AMOUNT OF DATA WE NEED TO DO RISK MANAGEMENT WELL?

Not particularly picking on Marcus, but in the article he uses the common complaint, “We lack the data to do risk management well.”  This mantra is repeated to the point where I’m blase’ about it.  But for some reason, this sentence really jumped out at me this time for two reasons.  It made me ask:

1.)  How do you know we don’t have the proper amount of data?

2.)  Can we even define “well” (i.e. what “good” risk management is) yet?

I really don’t know that the industry, especially concerning IT risk, is mature enough to really conclude that we don’t know (in the case of the former), nor that we can define (latter), conclusively.

PLAYING THE CONTRARIAN

Just because I’m feeling kind of zany this morning, let me suggest something.  Maybe there actually is lots of evidence out there for us to use.  Maybe:

1.)  It’s just that we don’t have particularly good models that provide context.

2.)  When that evidence isn’t an obvious phenomena that lends itself to easy measurement, we throw our hands up in disgust and fall back on “lack of data”, “can’t quantify risk”, “best practices work just fine” or any other number of arguments, no, excuses we use to justify our inability to be precise about the past (more or less the present or future - apologies to Niels Bohr).

IT’S IN THE WAY THAT YOU USE IT

Now I actually am happy to acknowledge that we don’t have enough data to be precise.  You, me, even smart guys like Marcus and Bruce - we’ll never be able to “engineer” risk management.  But you know what?  Neither can Insurance companies.  Sure, there are plenty of places where they have enough data to apply a traditional frequentist approach to risk valuations.   But there are plenty of times Insurers actually insure and they don’t have centuries or decades of data.  There are plenty of times when they rely on the “estimates” of subject matter experts.  There are many times they have enough information to be accurate rather than precise, and that’s good enough for them.

For that matter, it’s worth noting that there are plenty of scientific disciplines that have to deal in imprecise prior information, or evidence that’s fraught with uncertainty (what Ranum calls “squishy”, and what I’ve heard real honest to goodness physicists call “noisy”).  Unfortunately, we’re going to be like them.  Until we can read minds and predict the future, there will always be uncertainty in our measurements and posterior conclusions.  The trick is in how you deal with it and express it.  And while I really don’t know how much time Marcus or Bruce have really spent in the deep end on the subject of risk and its management - I have seen people doing brilliant things around risk (though they just aren’t mainstream).  Whether the tools are Bayesian methods, Monte Carlo engines, reductionist models of complex problems, there are risk analysts trying to deal with the problem.  These analysts are applying scientific method(s) and developing reasonable approaches to a very complex problem.  There are people trying, and our body of knowledge is growing, growing well beyond “gee, I haven’t got an obvious solution so I’ll blame it on lack of data”.  Heck, I’ve seen readers of this blog suggest Douglas Hubbard’s book in other security forums!*

I’VE GOT YOUR DATA RIGHT HERE…

But we don’t have enough data?  I have to ask, how much more do we need?  I mean crikey, JPMC just visited our ISSA chapter claiming, like, a bajillion events an hour.  There’s not one, but several companies out there that will want to tell you about how they have deep “insight” into the attacker community.  The boundaries of IT Risk losses are pretty well established by events that happen to public companies.  We have pretty mature testing/assessment tools and methodologies now that help us test our ability to resist the force an attacker can apply to us.  So what part of the Threat Landscape, Asset (Controls) Landscape, or Loss Magnitude landscape is too incomplete (and what are you doing to find the information you need)?

SO WHY DO WE FAIL?

Which brings me to a final, somewhat depressing conclusion.  Maybe there’s data, and maybe we’re starting to see the means to use it.  But in the end I do have to agree with Marcus that the vast majority of the infosec world *is* doing a really, really bad job with regards to “risk” and “risk management”.  The majority of people I know consider GRC to be a cruel, expensive joke.  Risk Assessment Methodologies tend to be built on the faulty premise that if we create a repeatable process, our measurements and conclusions will magically become accurate and wise.  Risk models tend to be factors loosely measured by ordinal scales and then somehow “multiplied” together to create a relatively meaningless qualitative value.  The State of the Union here is not good.  But after reading such a superficial treatment of an important and complex subject, I am left wondering if Bruce and Marcus were the right people to write about risk management in a mainstream publication.  As Inspector Callahan says, “A man’s got to know his limitations.”

===============================

* Speaking of which, if you want to do one cost effective thing to address your uncertainty - go find Douglas Hubbard’s book. It’s even got a nice recommendation from Peter Tippett.  The book is called “How To Measure Anything” - the title sounds rather hyperbolic, but there are good techniques in it we can use to identify useful information and refine our ability to frame that qualitative information into quantitative values. The key is how Hubbard has you deal with your uncertainty.  For those of you who are more scientific minded and want to dig deep into the subject, I have on good authority that E.T. Jaynes “Probability Theory, The Logic of Science” is a rather under appreciated work.

Tooting our own horn on Monday morning, the excellent Thinking Problem Management blog gave us their coveted “5 pineapple” rating!

In your face, RISKS Digest!

Several folks (Hi Daniel, Brent, David!) sent email & twitters asking us our opinion on a Dark Reading article called “Why Risk Management Doesn’t Work” which if you click on the link should come up for you after seeing someone’s advertisement for a few seconds.

I’m assuming the author wants us to read the title as “Things to Look Out For in Performing Risk Analysis” and not “Risk Management is Folly - Stop, Stop, Stop!” The former is fine, the latter isn’t supported by the evidence presented by the subjects of the article.
The subjects of the article are a good study from Wade Baker & Co. at Verizon, and a report from RSA’s Security for Business Innovation Council. Let’s take a look at each of these and examine why what they’re saying might contribute to poor risk management, shall we?

1.)  THE VERIZON REPORT

The Verizon report is an analysis of some 530 forensic investigations their company performed.  It is well worth your time as it’s chock full of interesting information.  As it relates to the Dark Reading piece, a coarse summary would be that “likelihood” is “different” for different people and so you can’t use the same “likelihood” across different industries.

Distilled through the lens of FAIR:

“different threat communities may be applicable based on Probability of Action factors which include: Value, Level of Effort and Risk (of Getting Caught).”

Or, even further distilled and in the words of my six year old son,

“Duh-uh”.

With regards to what I assume is the purpose of the article (What Doesn’t Work in Risk Analysis) this concept  seems just to rehash the old GIGO argument regarding risk analysis.  Great.  Can’t argue with that, nor it’s corollary QIQO (quality in, quality out).

But let me ask you -  is this really a problem common in your analysis?  Did reading this article make you go “Crap, we’ve been using data normalized across multiple industries in our analysis! They’re all wrong!”  Or have you already been accounting for the unique value proposition your company has to the specific threat community you’re worried about?  See, maybe I’m just not your average analyst, but even in my NIST/OCTAVE days, this has *never* been an issue for me.

Let me be specific, this is not a problem with Verizon’s very cool report.  It’s just that I don’t see what the big deal is.  This article is starting to feel like someone is running through the motions, trying to play the ” a crazy title gets people to read a boring article” game.

Speaking of cool reports - You know what would be cool?  I think it would be interesting to see is the quality of these companies’ “risk management process” established using good criteria,  and then correlated to the frequency and magnitude of real-world losses across the aggregate sample.  In other words, can we establish evidence that strong risk management practices not just reduce “risk” but also reduce actual incidents.

2.)  THE RSA COUNCIL “EXPLORES WHY LEGACY METHODS OF EVALUATING INFORMATION SECURITY RISK DON’T WORK IN TODAY’S CONNECTED WORLD, IN WHICH ANY NEW BUSINESS INNOVATION INHERENTLY CARRIES SOME LEVEL OF RISK TO INFORMATION.”

This report from the RSA council puts forth a seemingly obvious proposition, that risk must be balanced by reward.  Why is this news?  Now as I read the article it’s not clear if:

• The RSA Council is claiming that the CISO’s office should be the ones determining reward.  Absurd.

or

• Businesses aren’t doing a good job at determining risk and reward.

Let’s go with the latter.  So I’m pretty sure (good) businesses do a good job at estimating reward.  Businesses I’ve been a part of?  We LOVE(D) estimating reward.  We don’t tend to start projects all willy-nilly. No we tend to be careful to identify the size of the market and what it will cost to address the market.  So what could the problem be that this RSA council is trying to address?  Maybe it has to do with something like the following:

Yesterday, I got a demo of an IT-GRC application that shall remain nameless.  It seemed to be very good at the “C” bits - lots of information on regulations and expectations and even what sorts of controls would answer the regulations (which is goofy, but we’ll have to talk about that later).  It also gave you the ability to build workflow quite nicely.  But it measured NOTHING.  There really was no observable “G” and “R” was really Medium X Low X Low = High sorts of stuff.  So let’s use this relatively expensive tool as evidence of what your average CISO is armed with going into a Risk/Reward sort of meeting.  I imagine a nice board room with wood-grain paneling and glass bowls filled with little chocolate covered mints designed to give everyone involved in the meeting (CEO, CFO, CIO, CSO, VP S&M, etc…) a little sugar rush when needed and fresh breath.  The conversation goes a little something like this (apologies to Rich):

Business Guy Who Wants to Make Money Because That’s What Businesses Do: Based on market studies, we believe that initial gross revenues from the new product and technology rollout will be eleventy gazillion dollars based on a 37% market penetration in Scandinavia, alone.

CSO: Well now, we have a likelihood of “High” and a “C” impact of Medium, and an “I” impact of Low, and an “A” impact of “High” and because we are a (bank/hospital/retailer/basically any business that breathes anymore) we weight “C” by a factor of 2 - we multiplied those all together and got a “High”.

So can you guys delay the product rollout by 9 months and give me a bunch more money that’s not in the budget so that I can get this thing down to a “Medium”, please?

Again, I just don’t see the problem with Information Risk Management being that our businesses have no idea what the rewards of business might be.  Now maybe we need get a seat in that boardroom just to be able to talk about our “Mediums”, sure.  And maybe we’re infantile in our ability to describe our problem space.  But I cannot fathom that “Risk Management Doesn’t Work” because businesses haven’t been considering “reward”.

WHY RISK MANAGEMENT MAY  NOT BE WORKIN’ FOR YOU

Two meta-categories of causation:

• No skills

and/or

• No resources

Any ancillary “cause” can be mapped to one of these categories.  You could have significant resources but crappy models, and have conversations like our imaginary CSO, above.  You could have really good models and people trained and motivated to use them, but scarce time & money, so no conversation happens.

Now my question for you is - which does it make sense to acquire *first* to solve the “Why Risk Management Doesn’t Work” problems, skills or resources?

We’re frequently asked what we’re reading and what we like in blog posts, so here are some interesting things that hit our RSS readers that you may have missed:

COBIT rivals ITIL from The IT Skeptic

“Everyone is tiptoeing around the fact that COBIT offers a significant competitive body of knowledge (BOK) to ITIL. Sure ITIL goes into more depth in places, but to say COBIT sits over the top is to grossly understate the overlap. COBIT extends a long way down into the “how” and it does it with an intellectual rigour that ITIL lacks.”

Interesting stuff that.  A detailed mapping might help some folks.  Either way, the good news for those keen on understanding risk management is that governance metrics, done right, allow us to understand a part of that “capability to manage risk” we’re always looking for.   Assurance, verification and the acquisition and interpretation of knowledge is king.   Speaking of which….

How To Tell When “Nothing Happens” by Pete Lindstrom

“…problem is that, it isn’t really true that “nothing happens” when you employ some specific security control to prevent an exploit. Not only that, but even when it is difficult to collect data on what didn’t happen, one can devise experiments to tell how frequently that nothing occurred.”

Good analysis is all about the uncertainty.   Speaking of accounting for uncertainty…

Assets Good Until Reached For by Gunnar Peterson

“If you have a 100,000 dekstops or 100,000 servers it hard to manage. You will need to automate and to do that you need to abstract, but you should also realize that its a drawing on a whiteboard not reality. You need abstraction assurance.”

And there’s the trick.  We might call “abstraction assurance” an analog to “confidence” or “uncertainty” in certain priors (metrics) or posteriors (calculated values based on those metrics).  The stronger that abstraction assurance is, the less uncertainty we have in our knowledge and the better our ability to create wisdom from that knowledge (you know, make decisions).

Epstein, Snow and Flake: Three Views of Software Security by Adam Shostack

Adam’s focus is on software security, but the discussion here can be abstracted out into the broader realm of risk management quite nicely.

Two-thirds of firms hit by cybercrime from Security Focus

The US DoJ says that in 2005 (there’s some timely data) 2/3 of their surveyed firms detected at least one cybercrime.  “Cybercrime” is “classified … into cyber attacks, cyber theft, and other incidents.”  Pretty general.  Also from the report:  “Computer viruses made up more than half of all cyber attacks.”

(That sound you hear is me tapping my forehead lightly on large iron object)

Lessons Learned from “Personal” Risk Management By: Christopher Daugherty

“This process is what I call “personal risk management.”  All of us have done it and will continue to do so.  Why is it, then, many companies have ignored following similar principles with the on-going health of the business?  This is a debate with many different answers so I ask you to select the best answer for your employer:

a) Have not ignored as this keeps me awake at night!

b) Please restate the problem, I cannot hear well with my head buried in the sand.

c) We passed our SOX audit so we checked this off the list!

d) We are informed of the challenge but we have a business to run and profits to make

e) Is this what internal audit and risk management has been telling us?”