This post is big, too big. Alas, dear reader, you must serve as editor.
-
In a previous post, I wrote,
This suggested to me that players, people, will adapt to a structure over time, and, as I quite often see out there in the world, will likely learn ways to game the system. This adaptation to, or gaming of, the system can lead to new or unintended outcomes, perhaps beneficial, perhaps otherwise. In response, the structure will be modified to incorporate these behaviors and outcomes, perhaps encouraging them, perhaps counteracting them. And so on.
(I must say that John Barrow’s “The Artful Universe” just sprang to mind, a book that ventures an explanation of why things are the way they are.)
Thinking of gaming the system brings security to mind. Security generally seems to follow this pattern of tweaks – attacks get better, and defenses are modified accordingly. However, when a situation of radical change occurs, the instability that results can make security objectives quite difficult to meet, even if defined objectives exist. After all, locking down a system generally requires that the system have some stability and structure. Security itself is structure, and instability breaks down structure. So, radical change can require a shift from tweaks to a wider (re)building of a security structure as areas of this new system solidify.
Ok, this seems obvious. But, why not look at some examples?
Since the economy is on people’s minds these days, I thought these excerpts from Michael Lewis’ “Liar’s Poker” about Wall Street in the 1980’s would serve as the first, and perhaps best, example.
[...]At a rare Sunday press conference, on October 6, 1979, Paul Volcker announced that the money supply would cease to fluctuate with the business cycle; money supply would be fixed, and interest rates would float. The event, I think, marks the beginning of the golden age of the bond man. [...] Bond prices move inversely, lockstep, to rates of interest. Allowing interest rates to swing wildly meant allowing bond prices to swing wildly. Before Volcker’s speech, bonds had been conservative investments, into which investors put their savings when they didn’t fancy a gamble in the stock market. After Volcker’s speech, bonds became objects of speculation, a means of creating wealth rather than merely storing it. Overnight the bond market was transformed from a backwater into a casino.[...]
The mortgage trading desk evolved from corner shop to supermarket. By increasing the number of products, they increased the number of shoppers. The biggest shoppers, the thrifts [savings and loans], often had a very particular need. They wanted to grow beyond the limits imposed by the Federal Home Loan Bank Board in Washington. It was a constant struggle to stay one step ahead of thrift regulators in Washington. Many “new products” invented by Salomon Brothers were outside the rules of the regulatory game; they were not required to be listed on thrift balance sheets and therefore offered a way for thrifts to grow. In some cases, the sole virtue of a new product was its classification as “off-balance sheet.”
[...]Demand now exceeded natural supply. Huge pools of funds across America were dedicated to the unbridled pursuit of risk. Milken and his Drexel colleagues fell upon the solution: They’d use junk bonds to finance raids on undervalued corporations, by simply pledging the assets of the corporations as collateral to the junk bond buyers. (The mechanics are identical to the purchase of a house, when the property is pledged against a mortgage.) A take-over of a large corporation could generate billions of dollars’ worth of junk bonds, for not only would new junk be issued, but the increased leverage transformed the outstanding bonds of a former blue-chip corporation to junk.[...]
Washington Irving, so many years ago, best summed up how this sort of thing ends, words to which we can all relate today.
Could this delusion always last, the life of a merchant would indeed be a golden dream; but it is as short as it is brilliant. Let but a doubt enter, and the “season of unexampled prosperity” is at end. The coinage of words is suddenly curtailed; the promissory capital begins to vanish into smoke; a panic succeeds, and the whole superstructure, built upon credit and reared by speculation, crumbles to the ground, leaving scarce a wreck behind:
“It is such stuff as dreams are made of.”
Which takes me on a bit of a diversion…
When people discuss the economic situation of today, I sometimes notice a common mistake, a confusion of the representation (e.g., money) with the reality (e.g., productivity). We as people are great at building representations of the world to suit our purposes. For example, we build a visual representation of the world using our eyes and brain in order to assist with our navigation of this world. That visual representation is not reality, it is a representation of a small slice of reality, that slice useful to our moving around in the world, generated by our brains from inputs gathered by our eyes. However, if that representation becomes very distorted, if we can no longer make out objects blocking our path, we start to bump into things, we fall down, we hurt ourselves. Our distorted visual representation no longer adequately reflects reality for the purposes to which that representation was to be used.
Perhaps a more relevant example, say you have a map of your town depicting all the roads in your town. That map is one representation of your town, and it can be used to assist in the navigation of your town by road. However, you would never confuse tracing your finger along some path on that map with actually walking the streets of your town (i.e., “the map is not the territory“). Drawing a new road on that map does not magically create a new road in your town. And, if that map depicts roads not actually in your town, its usefulness for navigation is limited – you might just throw it away.
Coming back to the economic news of today, one way to think of money is as a representation of productivity. We created this representation to facilitate trade, savings, and investment, but, in order to be useful for these purposes, the representation must reflect the reality of productivity, much as our visual representation of the world must reflect the physical reality around us to facilitate our navigation. Money is not productivity any more than a map of your town is your town.
Now, people are good at gaming the system, and the system of money has been no exception here – people have learned how to game the system of money. Money was much easier to manipulate than the productivity it was supposed to represent, and gradually this property became heavily abused, rather than just used, by people. At some point, people lost sight of the reality for the representation, and the accumulation and growth of the representation became a goal unto itself. Productivity was left in the dust, like drawing a million roads on the map of your town but never building any of them. The representation was distorted away from the reality, and that distortion has now become so great as to severely damage the usefulness of the representation for its purposes.
(When you hear discussions of how to fix the system, what is being debated is how to remove the distortion and limit it in the future. Assuming the current system is to be retained, there seem to be two primary ways to remove the extreme amount of distortion from what I gather from others: 1) devalue the money, thus reducing the debt load denominated in that money to a level commensurate with realistic future productivity estimates, or 2) default on debt, thus reducing the debt load to a level commensurate with realistic future productivity estimates. (I am ignoring those caught up only in the representation and missing the reality, but decisions could certainly be made to go further in this direction as well.) Whatever happens, the result is, and will be, painful for many, and so the other topic of conversation is how best to limit the hurt.)
All of this brought to mind an interesting point with regards to design versus implementation flaws in light of representations and their purposes:
- A design flaw causes a system to misrepresent reality such that the objectives of that system are broken for all implementations of that design.
- An implementation flaw causes a system to misrepresent reality such that the objectives of the system are broken due to a mistake in implementing the design rather than a mistake in the design itself (and so only the mistaken implementation is broken).
Also, as I journey down this path, I am reminded that exploitation of the human factor has been a recurring topic in this blog. Attacks of this sort often boil down to taking advantage of a vulnerability created by a misrepresentation of people in a system. Which brings to mind three concepts utilized in systems and their relation to the people factor:
- Regulate. This attempts to restrict the human factor by trying to limit what people can do.
- Audit. This attempts to learn about the human factor by providing the ability to know what people do.
- Educate. This attempts to adapt the human factor by teaching the reality to comply with the representation.
Much like a self-help book, these bullet points make the world seem way too simple. For example, there may be a complex balancing act, overlap, and feedback between these three concepts, as well as, between these three, risk, productivity, and resources. Nevertheless, the abstraction seemed useful to my purposes. 
Ok, enough of this branch, let me get back on track. What follows are some examples of games and adaptations taken from this blog’s history.
Evolution seems the logical place to start in this travel through some prior posts, which leads us to an example found here.
Anyway, ignoring the multitudes of other factors around this sort of evolutionary understanding, it can be said that at least some part of our brains evolution seems like the result of a bit of an arms race. Our brains evolving more and more powerful capabilities for impressing potential mates (offensive capabilities) while at the same time evolving more and more powerful capabilities for selecting from potential mates (defensive capabilities). It could be that we are a product of “our own” attempts to secure reproduction. Cool beans.
(I can’t help but think of Geoffrey Miller’s “The Mating Mind”.)
This brings to mind some thoughts from an old ramble,
Moving on from just beauty, marketing and sales are about trying to influence people to spend their limited resources (e.g., time, money) on certain products (e.g., goods, services) over other products. This done is [yes, a typo in the original] through many means, both overt and covert. It preys upon our base programming and our learned experience – things like social value, liking someone, or even just wanting to seem consistent – to get us to do things the marketing and sales people want. (Remember this?) Beauty helps with things like being liked and social value. It also helps make objects and other people more desirable. Think of that attractive model sitting on that car for sale – regardless of how blatant the use of the model is, your brain makes associations between that model and that car. And, as noted above, I can think of the increased attention I get when I am out with beautiful people – my social value is increased just by association.
(Robert Cialdini’s “Influence” is right up this alley.)
And its follow-up,
The thing is, all our base programming and all our experience can be used against us. From the power of beauty to inaccurate perceptions of risk, it could be that security mechanisms fail to meet their objectives because, well, real people are involved. When security does not hit upon how real people act out there in the real world, it seems to miss a big chunk of risk.
Which leads us to discussions of ID (age) checks.
In order to up the probability of success, the ladies employed two secondary techniques. One technique was having the most adept social engineers of the group conduct the primary interactions with the ID checker. The ranking of such skills was subjective, but it generally came down to being attractive and social. The point of this was to get the checker in the frame of mind of wanting to allow the ladies to enter, which also implied getting the check [yes, another typo] to want to accept the IDs being presented. This was also used to avoid any signs of nervousness from the group and get them looking the part. The other technique was just memorizing the details on the ID and being ready for small talk with the ID checker (that applied to everyone, not just the primaries used in the first secondary technique) – such interactions were rare, but did happen from time to time. With these two simple measures in place, their impression of the effectiveness of the attack was at about 95%.
The crypto world tends to be conservative, but it moves nonetheless.
RSA has come up here.
Also via Metzger’s cryptography mailing list, five years to the month after Lucky Green spoke out and said no more to 1024 bit RSA keys, the gap continues to close on 1024 bit RSA keys.
As has password hashing.
Even with salts, the traditional Unix crypt function is quite dated and may not be a good choice for generating password hashes. The tiny salts, short password length, optimizations, current processing speeds, etc. all combine to make crypt function generated password hashes quite susceptible to password cracking efforts. The two common alternatives are the MD5 crypt function, which uses the MD5 hash algorithm as its underlying crypto primitive and incorporates a salt, and the blowfish crypt function, which uses the blowfish encryption algorithm with modified key schedule as its underlying primitive and incorporates both a salt and an interation count. The blowfish crypt function is considered to be the latest and greatest crypt function that is widely deployed.
As is the search for a new SHA.
Backing out to a common IT task, a recent post illustrates the vulnerability followed by patch stream many of us know all too well.
USN.
[...]
FreeBSD VuXML.
[...]
FreeBSD Security Advisories.
Like patching vulnerabilities (and segueing back to people), when we recognize the games of others, we can adapt to them.
Anyway, this reminded me of Cialdini’s Influence. The attacks of influence are often carried out beneath the radar of the person being attacked. The attacker triggers automatic responses in the person to influence their decisions/behavior, and the actions that hit these triggers go unnoticed at a conscious level by the person being attack at the time of attack, which results in the person being attacked not properly recognizing the level of influence coming from the attacker. Once a person is aware of triggers and/or able recognize attempts to pull triggers, a person can work to mitigate the influence of triggers and/or the responses to triggers.
(While I note Cialdini there, Paul Eckman’s “Emotions Revealed” also seems relevant.)
And now, dear reader, as you are likely reaching the end of your rope, we shall finally finalize this regurgitation with the finale of a prior set of rambles for our finish.
First off, we need a threat model. We need to figure out what we want to protect, its value, the potential attacks, the likelihood of those attacks, and the potential damage of those attacks. Determine the risks, and then mitigate them. Very important here is figuring out who needs to be held responsible for what mitigations and countermeasures. And, this threat model has to be reviewed periodically to keep it up to date. The assets that need to be protected change, the way business is done changes, the risks change, the mitigations change. Security is not static.
With that (and building that is certainly not trivial), we have these people, you, me, our parents, that are part of this threat model. They pose risks, and they help to mitigate risks. We need to minimize the former and maximize the latter. To do this right, I think we need people to feel responsible for security. To build this sense of responsibility, we need these security responsibilities audited and we need effective training to convey and reinforce these responsibilities – the combination of these two may be a linchpin to people security.
So, we talk to people about our threat model. Not only do we teach it, but we get feedback on it. And, we make sure everyone understands that threat model, and their place in it. To do this, we bring vivid examples from security audits into our security education, which help to build security training programs that provide exactly that which we learn the most from, experience.