Assorted Notes on Defense and Exploitation

[spender]

As I mentioned at the RSA Europe panel on "the History and Future of Binary Exploitation", we have been seeing and will continue to see a shift in importance from the discovery of vulnerabilities in individual pieces of software to the discovery of vulnerabilities in mitigation techniques. The effectiveness (and cost-effectiveness) of the protections pioneered in PaX should be clear to everyone by this point, even if the clarity of this situation is a bit muddled by poor implementations/design decisions on certain OSes.

It was for this reason that I was a bit surprised by the presentation by Andreas Bogk at 27C3 recently entitled "Defense is not dead." I of course don't argue with this title, though I'd go further to say that it hasn't been dead for the past 10 years, if you've been paying attention. I disagree with the conclusions of the presentation though, and some of the statements made in it bothered me. A video of the presentation is available at: http://achtbaan.nikhef.nl/27c3-stream/r ... ot%20dead/ . Andreas is definitely very bright, but discussing the solution to our security problems in terms of removing all the exploitable bugs through formally-verified compilers and new languages/processors/OSes seems rather misguided.

Defense will die of old age before normal users will see security problems evaporate from these technologies. Our real-world experience should have sufficiently demonstrated to us that ridding the world of vulnerabilities will never happen (not to mention all the legacy code still laying around). Formal verification also has the real potential to introduce a sense of false security. How will users have knowledge of what was proven where? The verification is only as good as the proof. Does any long-term adoption plan exist? How would we (or is it even possible to) transition to such an extreme solution? Is it possible to solve the problem of legacy code? Can you run an old firefox binary on this new system without having your data stolen out of it via an exploit? There are all kinds of other societal factors to consider, like the fact that users are generally against alteration of their computing experience in a significant way (see the backlash against the Vista UAC). Barring some reality-based answers to these questions, I'm inclined to think of this approach more in terms of concept cars or fashion shows: the point isn't to take the "solution" wholesale, but to find what bits and pieces you can integrate into current solutions in a more progressive way.

We continue to see vulnerabilities in MS's implementation of PaX's PAGEEXEC/ASLR features (or vulnerabilities in other vendors' use of said mitigations due to certain design decisions and compromises). It's surprising actually that this many years into having this functionality, large vendors continue to apparently fail at implementing any kind of QA along the lines of "don't ship out any code without /DYNAMICBASE," which has lead to nearly all of the "ASLR bypass" stories that have cropped up in the past year or so.

It's important that the vendors take the initiative here, because with the trends in security going the way they are, the self-interest of the industry prevents it from being benevolent in reporting these kinds of mitigation vulnerabilities. There's a difference between killing a single vulnerability, which is probably found via fuzzing and which another researcher has likely also found, and killing a mitigation vulnerability that may make the next 20 vulnerabilities discovered unexploitable (or only unreliably exploitable). I've already heard of this kind of behavior taking place, and it's unfortunate and unacceptable that money/job security/industry security is given greater priority over actually fixing security problems. But this is all further evidence in what I've mentioned to others as an industry knowingly in its death throes. The tendency to ignore the latest OS/architecture trends and the latest mitigations in exploitation research is yet further evidence.

On the topic of mitigations, it's important to make the distinction between things like PAGEEXEC/SEGMEXEC/MPROTECT/ASLR and EAT address filtering. EAT address filtering has more in common with things like Comodo's Memory Firewall product that claimed protection against ret2libc (see: http://forums.comodo.com/frequently-ask ... 237.0.html) but simply contained a weak protection against canned ret2libc exploits against specific APIs. With any knowledge of the method Comodo was using, through some minutes of reversing, it wasn't difficult to modify a ret2libc-style exploit to still work. In November, Skylined published a bypass for EAT address filtering, which he wisely called a "pseudo-mitigation" (see: http://skypher.com/index.php/2010/11/17/bypassing-eaf/). A side-note about the comments to the article: DRx are privileged registers, so access to them by unprivileged code triggers an exception (i.e. you cannot just clear DR7 to evade detection). GD thus has nothing to do with this -- it would only cause privileged access to DRx to fault as well. There are other caveats associated with GD that generally makes it useless for anything security related, but people always like throwing it out there as evidence that they read the Intel manuals or something.

We need to be likewise careful when discussing "sandboxes." It's a failure that we have discussions about software like Adobe Reader that boil down to: "Person 1: Adobe Reader X has a sandbox. Person 2: A sandbox? that's great!" The implementation of the sandbox matters. This is why information like that contained in Mark Dowd's posts: http://blog.azimuthsecurity.com/2010/05 ... rview.html are important, though it would be nice to have this kind of information more readily available to users of the software and without the bias of the vendors themselves. Preventing the execution of arbitrary code or other control flow modifications has benefits for sandboxing as well. If you can guarantee that an attacker can't just issue arbitrary system calls (in order to exploit a kernel-level vulnerability as part of his/her exploit chain) then it doesn't matter so much that the sandbox in place has no syscall-level filtering.

Defense isn't dead, but it's not easy to do (properly), especially under the constraints Microsoft and others have to operate under. It's too bad there aren't more people in the industry interested in defense -- if there were, perhaps we could all work together to put ourselves out of jobs!

Accepting applications,
-Brad

[andreas]

Brad,

with all due respect, but from my point of view, it is PaX, or to be more precise, the whole approach of W^X and address space randomization, that is misguided. It should be obvious that preventing a bug from happening is much more efficient at preventing exploitation than making life harder for the exploit author. I predicted way back in 2005 on the ph-neutral that this approach will have limited effect. Time has proven me right, especially with the advent of ROP. And any boring and old information disclosure of the kind we used to ignore can be what breaks your ALSR these days.

I predict the same will happen with the Adobe sandboxes. People will break them. How about doing it right? With the amount of money at Adobe's hand, it should be possible to write a PDF renderer that's not a complete and utter desaster. You don't need a sandbox when your code is secure. In fact, it is DEP and ALSR and sandboxes that illustrate what's wrong with defense today!

Besides, it's just not the security. I'm fed up with crashing software and crashing computers. Sure, PaX may make writing an exploit harder. But I still get to reboot my machine when a bug in the kernel is triggered.

cheers,
Andreas

[spender]

Hi Andreas,

If you could elaborate on a few things, we can have a more grounded, evidence-based discussion, instead of devolving into meta-security discussion about what I think you believe or vice versa:

How has "the advent of ROP" changed things, when it itself was essentially a renaming of ret2libc, was coined 8 years or so after PaX developed ASLR, and yet generally ignored the existence of ASLR?
https://www.blackhat.com/presentations/ ... amming.pdf (no matches for ASLR)
http://cseweb.ucsd.edu/~hovav/dist/rop.pdf mentions ASLR only twice in 41 pages:

Roughly speaking, most of this activity falls into
two categories: efforts that attempt to guarantee the integrity of control flow in existing
programs (e.g., type-safe languages, stack cookies, XFI [Erlingsson et al. 2006]) and efforts
that attempt to isolate “bad” code that has been introduced into the system (e.g., W^X,
ASLR, memory tainting, virus scanners, and most of “trusted computing”).

Address-space layout randomization (ASLR) is another orthogonal defense. Typical
implementations, such as PaX ASLR for Linux [PaX Team a], randomize the base address
of each segment in a program’s address space, making it difficult to determine the addresses
in libc and elsewhere on which return-into-libc attacks rely. Linux implements ASLR
on SPARC, but Solaris does not. Derandomization and other techniques for bypassing
ASLR [Shacham et al. 2004; Durden 2002; Nergal 2001] may be applicable to returnoriented
programming.

In what way outside of the flawed MS implementation (resulting in ASLR "bypasses" that just involved non-randomized libraries that I already mentioned) have the technologies of PaX had a limited effect in real-world exploitation? Do you have a list of Firefox/IE/Adobe Reader/Flash (post JIT-hardening) vulns that would be exploitable for code exec or stealing of user data under Windows 7 with EMET set to max security?

Since we're on the topic of Adobe Reader, do we have a rough estimate of how many lines of code it contains, or how many lines would need to be separated off into a privileged component? Of the real-world examples of compiler-assisted formal validation you gave in your presentation, I think the highest line count you mentioned was ~10,000 lines. I think you also mentioned MS threw 50 people at their hypervisor problem. Were they all involved in the same sense that developers attempting a similar rewrite to Adobe Reader would be involved? Do we know how many were involved in the addition of the sandbox to Adobe Reader? How well would the formal validation scale if it was being done against a codebase of 100,000 lines of privileged code? Do you have rough estimates on the time/manpower/monetary cost of that kind of validation?

Do you have answers to/thoughts on the questions I raised in the original post?

Formal verification also has the real potential to introduce a sense of false security. How will users have knowledge of what was proven where? The verification is only as good as the proof. Does any long-term adoption plan exist? How would we (or is it even possible to) transition to such an extreme solution? Is it possible to solve the problem of legacy code? Can you run an old firefox binary on this new system without having your data stolen out of it via an exploit? There are all kinds of other societal factors to consider, like the fact that users are generally against alteration of their computing experience in a significant way (see the backlash against the Vista UAC).

Do you view the system being developed as part of the DARPA grant as a viable goal, or as I mentioned, as something to pick ideas from for integrating into currently available systems? If so, how will we get everyone to use it? If not, how can you realize your security goals when only parts of it are hacked on to existing systems/applications? That is, your approach seems to be better only in an all-or-nothing case: if you don't actually remove all the bugs, then in a sense it hasn't advanced beyond competing incomplete solutions to security (which currently exist with known strategies to improve them further at low cost).

Finally, do you see any worth in raising the cost of exploitation?

-Brad

[PaX Team]

with all due respect, but from my point of view, it is PaX, or to be more precise, the whole approach of W^X and address space randomization, that is misguided.

just to be sure we're on the same page, you're familiar with the PaX docs, right?

It should be obvious that preventing a bug from happening is much more efficient at preventing exploitation than making life harder for the exploit author.

the problem with this in practice is that you either have to know where the bugs are or at least you have to know of all possible kinds of bugs to be able to develop prevention mechanisms for them (say, bounds checking against array bound violations, etc). now it so happens that noone knows how to do either (trust me, if you can demonstrate knowledge of either, you'll be very rich overnight). the best we have is that we know of some bug classes and have some usable prevention mechanisms against a subset of them. unfortunately for the defense side, users tend to value performance and convenience over security so even the existing prevention mechanisms cannot be all deployed in real life systems. that leaves us with the next best practical approach: ensuring that the environment is hostile to exploits. one way of doing this is demonstrated by PaX (i'll refer you to the docs again, although i plan to write more about this topic in the blog too) where we split up the 'exploit technique space' into subsets we can tackle with various techniques (some already exist, some will one day, some we don't know yet).

I predicted way back in 2005 on the ph-neutral that this approach will have limited effect. Time has proven me right, especially with the advent of ROP. And any boring and old information disclosure of the kind we used to ignore can be what breaks your ALSR these days.

if you take a look at those PaX docs you'll realize that your predictions were a few years behind the state of the art even back then . the various mechanisms developed in PaX a decade ago all have proven to be the right approaches for preventing or mitigating a particular kind of exploit technique. that a particular vendor's implementation is not good enough isn't a problem with the mechanisms themselves but the vendor and sometimes their ecosystem they have to live with. i find it somewhat amusing that you mention ROP as 'proof' given how it had been known as an exploit technique long before even PaX and how prevention/mitigation techniques had been developed something like 2 decades ago at least (yes, that was before the world at large knew what exploiting memory corruption bugs was all about). the reason you don't find these techniques deployed on a wide basis is something i'll have to blog about separately i guess (the PaX future doc tells the basic story though) but it doesn't mean we haven't known what to do about this exploit technique. as for ASLR, it's really obscurity that we use because its cost/benefit ratio is very low due to its low cost, not due to its high benefit . in other words, when it works, we got it for free basically, when it doesn't, nothing is lost that wasn't lost already.

I predict the same will happen with the Adobe sandboxes. People will break them.

actually, not counting implementation bugs, people didn't break anything, certainly not non-executable page based mechanisms or ASLR. what they did is that they had to find different ways to accomplish the same goals (ret2libc/ROP, finding info leaks in addition to 'regular' bugs, data only attacks, etc). it's actually proof that these mechanisms work in fact since they force the old ways of exploiting bugs out and drive exploit costs up (the rumor is that local kernel exploits against grsec cost almost as much as remote/client side windows exploits in the market). what will happen with (properly implemented) sandboxes is that people won't 'break' them, instead people will pair their PDF reader exploits with a kernel exploit (actually this already happened on 'hardened' systems for many years now). it's not hard to predict that this in turn will result in the defense focus shifting to kernel-land exploit prevention/mitigation techniques, something we've been doing for quite some years now and will most likely lead the pack again (in the past year or two i saw a few academic papers reinvent basic concepts of KERNEXEC for example).

How about doing it right? With the amount of money at Adobe's hand, it should be possible to write a PDF renderer that's not a complete and utter desaster. You don't need a sandbox when your code is secure.

as i said above, the problem is that noone can do it right. there's a certain level of complexity (it loosely correlates to the size of the codebase in question) above which it is seemingly not feasible for feeble human minds to produce bugfree software (with or without automated assistance).

In fact, it is DEP and ALSR and sandboxes that illustrate what's wrong with defense today!

if you're talking about the Windows implementation of these defense mechanisms then you're only complaining about implementation issues, not the concepts. if you have a problem with the latter, take a properly configured PaX setup and show us what is wrong there .

Besides, it's just not the security. I'm fed up with crashing software and crashing computers. Sure, PaX may make writing an exploit harder. But I still get to reboot my machine when a bug in the kernel is triggered.

it's called a tradeoff and you're free to get owned vs. having to restart your apps or box when under attack. based on a decade of experience, it appears to me that the world at large prefers the latter. that's not to say though that we shouldn't try to do better but the practical 'solution space' is extremely limited due to the above mentioned user expectations about performance and convenience.

PS: do you have a lock on your front door?

[mmiller]

Quite unsurprisingly, I agree with Brad and PaX team :) This is a good topic to discuss. Here's my take:

The ability to completely eliminate all vulnerabilities and prove their non-existence in the context of commercial native software is (and will likely remain) a pipe dream. That’s not to say that attempts to work toward this are misguided – far from it – rather, it is important to understand the practical limitations of such an approach. Software vendors are at a disadvantage when it comes to eliminating vulnerabilities because they must find and eliminate all vulnerabilities whereas an attacker only needs to find one that is exploitable (a point which has been raised many times). This imbalance means that software vendors must generally invest significantly more resources into staffing and tool development in order to find and fix vulnerabilities before an attacker uncovers one. This demonstrates that it can be a significant challenge for software vendors to scale their vulnerability finding efforts in a way that exceeds the aggregate capacity of attackers (whose collective budgets are likely to greatly exceed those of software vendors). In summary, it should be clear that the vulnerability finding investment cost for software vendors is much higher than an attacker’s and ultimately an attacker has more to gain from finding just a single vulnerability.

That’s just the economic side of the argument. The technical theories and approaches to finding and proving the non-existence of vulnerabilities are still immature at this point and are in relatively early stages of being applied to large scale programs (where large scale currently means just a single module in most cases). These theories and approaches are also in the early stages of being formalized to the point of being able to detect various classes of vulnerabilities. The important point here is that, today and for the foreseeable future, the tools do not exist for proving the non-existence of all vulnerabilities in large scale programs. In the long term some of this work will likely mature to this point – the research is certainly going in this direction. But how long will this take? And how much will need to be invested by the vendor? And will the threats we see today still be as relevant when these capabilities become available? These questions are challenging to answer.

At this point it should be clear why software vendors are interested in finding alternative methods of countering the threat posed by vulnerabilities. Ultimately an attacker is interested in achieving a return on the investment they have made into finding and developing an exploit for a vulnerability. In lieu of trying to find and eliminate all vulnerabilities a software vendor can instead focus their attention on making it as difficult and costly as possible for an attacker to develop a reliable exploit (hence exploit mitigations). This approach is quite attractive because the focus is placed on breaking exploitation techniques that are relevant to specific vulnerability classes rather than fixing a single instance of a vulnerability. This point is important because there are a limited number of exploitation techniques and the rates at which new exploitation techniques are discovered is much less than the rates at which vulnerabilities are found. This suggests that software vendors should expect to achieve a broader impact relative to their investment by developing or using exploit mitigations. Since the cost of developing new exploitation techniques is high, attackers need to initially invest much more in order to exploit a vulnerability which may decrease or remove their incentive to do so.

I wanted to briefly comment on implementation limitations of exploit mitigations. In practice the challenge with exploit mitigations is that they introduce new invariants or make subtle environmental changes that can be incompatible with legacy applications. This is especially true on Windows where there is an extensive amount of legacy software. As Brad and PaX team mentioned, PC users would definitely prefer that their applications work and run quickly rather than have some security feature enabled. The reason for this is obvious: an application that doesn’t run or runs slowly is easy for the user to observe (and get upset about) whereas the benefits of a security feature may never become apparent (especially since most exploit mitigations are not able to prove that an exploitation attempt occurred). This is why certain exploit mitigations are OptIn by default on Windows – a balance had to be struck between security, performance, and compatibility.

I agree that it’s very important for vendors to take the initiative when it comes to adopting mitigation technologies. This is one of the reasons why it’s valuable for software vendors to incorporate something like the SDL into their software engineering processes. The SDL in this case embodies knowledge about which exploit mitigation technologies are important and communicates this knowledge in the form of requirements (e.g. must link images with /DYNAMICBASE). I think part of the reason for the relatively slow adoption of OptIn mitigations features is that many software vendors are in the early stages of awakening to software security and it is currently difficult for them to objectively quantify the impact of enabling a mitigation feature. Assuming this pace quickens in the future it must still be observed that legacy applications will continue to have legacy mitigations – in other words, an application that uses the latest and greatest mitigations today may find itself out of date 5 years from now when new exploitation techniques come to light. This demonstrates one of the challenges of continuing to mitigate exploits throughout the lifetime of a shipped product (which may be supported for 10+ years).

Brad, I agree with your comments regarding EAF. One clarification to make is that the EMET user guide explicitly calls out mitigations like EAF and heap spray address reservation as pseudo mitigations (SkyLined adopted this terminology). Specifically:

"Please note this is a pseudo mitigation designed to break current exploit techniques. It is not designed to break future exploits as well. As exploit techniques continue to evolve, so will EMET."

Matt

[andreas]

How has "the advent of ROP" changed things, when it itself was essentially a renaming of ret2libc, was coined 8 years or so after PaX developed ASLR, and yet generally ignored the existence of ASLR?

Let's get that straight. ret2libc was basically known forever, but consisted in its original form of placing some arguments on the stack and returning to system(). ROP is insofar progress in the exploitation technique, as it allows to assemble arbitrary computations out of code snippets and thus execute arbitrary code, instead of calling just one single existing function. It is the way to work around noexec, not the way to work around ASLR. But still, ROP helps, because you're not limited to call system() as in the classical ret2libc, but instead can chop up almost any decently-sized block of code you can get your hands on into gadgets and call them.

The workaround for correctly implemented ALSR is to find an information disclosure bug that allows reconstruction of the base address for whatever you then turn into gadgets.

In what way outside of the flawed MS implementation (resulting in ASLR "bypasses" that just involved non-randomized libraries that I already mentioned) have the technologies of PaX had a limited effect in real-world exploitation?

Essentially, a combination of a write memory corruption and a read memory corruption is what's needed to beat all the combined technologies. As a thought experiment, consider perfect bugs, which would allow me to read from and write to arbitrary memory addresses, an arbitrary number of times. Do you really think there is any conceivable mitigation that could not be beaten?

I do not claim that PaX is useless, far from it. It makes life for the attacker harder. But I do claim that it it represents an evolutionary dead end in terms of computer security. It only addresses memory corruption issues, and it only makes it harder to exploit those, not impossible.

Do you have a list of Firefox/IE/Adobe Reader/Flash (post JIT-hardening) vulns that would be exploitable for code exec or stealing of user data under Windows 7 with EMET set to max security?

You're taking bets that none exist?

Since we're on the topic of Adobe Reader, do we have a rough estimate of how many lines of code it contains, or how many lines would need to be separated off into a privileged component? Of the real-world examples of compiler-assisted formal validation you gave in your presentation, I think the highest line count you mentioned was ~10,000 lines. I think you also mentioned MS threw 50 people at their hypervisor problem. Were they all involved in the same sense that developers attempting a similar rewrite to Adobe Reader would be involved? Do we know how many were involved in the addition of the sandbox to Adobe Reader? How well would the formal validation scale if it was being done against a codebase of 100,000 lines of privileged code? Do you have rough estimates on the time/manpower/monetary cost of that kind of validation?

I've seen a class diagram of Adobe Reader, reverse-engineered from the binary and the RTTI. But we're not talking about separating off some piece of that code and verifying that. We're talking about:

1. Throw away all old code
2. Choose an implementation language that doesn't allow the programmer to write code that will lead to memory corruption
3. Prove correctness of the language implementation and its runtime.
4. Write new code for reader

CompCert, the third project I've shown, is 100000 lines, btw. Language runtimes are usually on the order of 30000 lines of code, so that's feasible. The reader itself is probably quite a lot more code, but I do not even need to consider it in my calculation, since memory corruption issues are dealt with in the compiler and runtime. The key is to provide the programmer with an abstraction where he can't corrupt memory. So essentially, the cost is rewriting the application, plus some cost amortized through all users of that language.

Do you have answers to/thoughts on the questions I raised in the original post?

Sure:

Defense will die of old age before normal users will see security problems evaporate from these technologies. Our real-world experience should have sufficiently demonstrated to us that ridding the world of vulnerabilities will never happen (not to mention all the legacy code still laying around).

First of all, I'm not just concerned with normal users. Everybody is affected by the current disaster. Eventually, there will be no alternative to the more secure technologies trickling down to the normal user. Then, I do think that it is possible to completely eliminate whole classes of vulnerabilities. The fact that it hasn't been done yet only shows the limits of the attempts used in the past, not that it is impossible. Nobody would have thought it was feasible to prove the correctness of an optimizing C compiler either, until someone came and did it.

Formal verification also has the real potential to introduce a sense of false security. How will users have knowledge of what was proven where? The verification is only as good as the proof.

The possible fallacy when writing proofs is that the proposition you're trying to show doesn't actually mean what you think it means. The proof itself is machine-checked. However, with a good engineering of the overall solution, you will end up with concise theorems than can be reasonably peer-reviewed. Proofs don't replace code audits, they just make them significantly easier. I can eliminate whole classes of vulnerabilities by sufficiently convincing myself that the theorems say the right thing in the context of the code.

Does any long-term adoption plan exist? How would we (or is it even possible to) transition to such an extreme solution? Is it possible to solve the problem of legacy code? Can you run an old firefox binary on this new system without having your data stolen out of it via an exploit?

Transitioning between generations of systems has always happened. There are two ways of transition: keeping legacy hosts in their own isolated networks with data flow enforced by more secure machines, or running legacy code in emulators. I'm a bit careful with the latter solution in high security environments, cache effects could establish hidden channels from the vulnerable legacy code into the more secure systems.

But in general, I do not think that the solution needs to solve all problems of all users in order to make sense as a product. It's sufficient if it provides some fundamental features, but on a level of security so far unseen.

There are all kinds of other societal factors to consider, like the fact that users are generally against alteration of their computing experience in a significant way (see the backlash against the Vista UAC). Barring some reality-based answers to these questions, I'm inclined to think of this approach more in terms of concept cars or fashion shows: the point isn't to take the "solution" wholesale, but to find what bits and pieces you can integrate into current solutions in a more progressive way.

Well, I think both is the case. On one hand, there seems to be a demand for completely secure systems. On the other hand, using bits and pieces to improve is certainly a good strategy for legacy systems. Using a secure language for application implementation won't stop kernel exploits, but it will make the application more secure. Microsoft goes into this direction, note their efforts to move as much of their code base as possible to .NET managed code. There are efforts under way to make sure the VM is reasonably safe (as in: proving correctness).

Coming back to the other post:

Do you view the system being developed as part of the DARPA grant as a viable goal, or as I mentioned, as something to pick ideas from for integrating into currently available systems? If so, how will we get everyone to use it? If not, how can you realize your security goals when only parts of it are hacked on to existing systems/applications? That is, your approach seems to be better only in an all-or-nothing case: if you don't actually remove all the bugs, then in a sense it hasn't advanced beyond competing incomplete solutions to security (which currently exist with known strategies to improve them further at low cost).

i think the approach is viable. And yes, in th end, it only makes sense to throw away the legacy code.

Finally, do you see any worth in raising the cost of exploitation?

Certainly. Unfortunately, attackers have deeper and deeper pockets.

-Andreas

[spender]

Let's get that straight. ret2libc was basically known forever, but consisted in its original form of placing some arguments on the stack and returning to system(). ROP is insofar progress in the exploitation technique, as it allows to assemble arbitrary computations out of code snippets and thus execute arbitrary code, instead of calling just one single existing function. It is the way to work around noexec, not the way to work around ASLR. But still, ROP helps, because you're not limited to call system() as in the classical ret2libc, but instead can chop up almost any decently-sized block of code you can get your hands on into gadgets and call them.

Yes, I would like to get that straight, but unfortunately your view isn't supported by the evidence If you look back to 2001there was nergal's paper: http://www.phrack.org/issues.html?issue=58&id=4#article

The similar technique was shown in [5]. Instead of returning into a
standard function epilogue, one has to find the following sequence of
instructions in a program (or library) image:

pop-ret:
popl any_register
ret

Such a sequence may be created as a result of a compiler optimization of a
standard epilogue. It is pretty common.

This looks like what you're calling ROP to me The reference in [5] is to Tim Newsham's post in 2000:
http://seclists.org/bugtraq/2000/May/90
Or see this post from Ingo from 2005, discussing "Accidental Turing Machines" (that sound familiar? ):
http://www.mail-archive.com/linux-kerne ... 60270.html

In general, whenever people were talking about chaining ret2libc, they were talking about ROP. I hope next you won't say JOP is some new thing too, as it was already discussed in pax-future.txt from 2003 Just because the popular public exploits of the previous decade amounted to a system() ret2libc, it doesn't mean people weren't aware and publicly discussing more involved techniques. It's simply a matter of necessity, something you're still seeing even after the "invention" of ROP -- do as little as you need to do to exploit the process with the common protections applied to it. Even these modern "ROP" payloads do things like disable DEP or create an RWX buffer to execute the real payload in (see http://blog.metasploit.com/2010/03/late ... ssion.html).

I do not claim that PaX is useless, far from it. It makes life for the attacker harder. But I do claim that it it represents an evolutionary dead end in terms of computer security. It only addresses memory corruption issues, and it only makes it harder to exploit those, not impossible.

It's not a dead end if the PaX project isn't done yet. There are still lots of interesting things to do -- have you read pax-future.txt?

You're taking bets that none exist?

No, I was just curious if you had some evidence to support your claim that PaX technologies had only a limited effect. To my recollection, there were less than a handful that would have been possible against the Windows 7/EMET system I mentioned (I try to follow this kind of information). Going from every applicable Firefox vulnerability being capable of owning your system to less than a handful (and I think I may be being generous here), which likely won't be exploited in any sort of wide-spread driveby exploiting until long after your browser auto-updates -- that seems more to me like a huge success. Of course, then there's abominations like Java, that bastion of security, that makes for instant cross-platform/architecture compromise. Most people can get by without it these days, though (years ago when it was the "next big thing," that may have not been the case).

I've seen a class diagram of Adobe Reader, reverse-engineered from the binary and the RTTI. But we're not talking about separating off some piece of that code and verifying that. We're talking about:

1. Throw away all old code
2. Choose an implementation language that doesn't allow the programmer to write code that will lead to memory corruption
3. Prove correctness of the language implementation and its runtime.
4. Write new code for reader

CompCert, the third project I've shown, is 100000 lines, btw. Language runtimes are usually on the order of 30000 lines of code, so that's feasible. The reader itself is probably quite a lot more code, but I do not even need to consider it in my calculation, since memory corruption issues are dealt with in the compiler and runtime. The key is to provide the programmer with an abstraction where he can't corrupt memory. So essentially, the cost is rewriting the application, plus some cost amortized through all users of that language.

Ok, but arguably the people working on CompCert are far above your average PHP developer in intelligence There are many reasons why people choose certain languages, even long after we know how bad they are. So I'm more interested in how expensive that Adobe Reader rewrite would be. For the idea to be seen through to its completion, all those average PHP developers need to be converted too -- we can't have the world relying on a small (or smaller) group of super-developers -- the cost would explode.

The possible fallacy when writing proofs is that the proposition you're trying to show doesn't actually mean what you think it means. The proof itself is machine-checked. However, with a good engineering of the overall solution, you will end up with concise theorems than can be reasonably peer-reviewed. Proofs don't replace code audits, they just make them significantly easier. I can eliminate whole classes of vulnerabilities by sufficiently convincing myself that the theorems say the right thing in the context of the code.

What I meant to imply here was that all ranges of companies have obvious self-interests that result in them making "short-cuts," especially when their customers are none-the-wiser. If annotations aren't provided to prove certain properties about the code, then those properties won't be validated, yes? So my interest wasn't in whether the proof does what it says, I understand that that's checked. I'm more concerned about what the actual proof says (what properties have been validated) and how we'd be able to know that information.

Transitioning between generations of systems has always happened. There are two ways of transition: keeping legacy hosts in their own isolated networks with data flow enforced by more secure machines, or running legacy code in emulators. I'm a bit careful with the latter solution in high security environments, cache effects could establish hidden channels from the vulnerable legacy code into the more secure systems.

Even if you're emulating though, you're retaining compatibility with some old, insecure code. Sure, now it can't feasibly compromise your entire system (we hope) but an owned browser is nearly as bad these days. So I'm not concerned with covert channels, but with how you prevent exploitation of legacy code without something like the techniques in PaX.

Certainly. Unfortunately, attackers have deeper and deeper pockets.

Well, that should be qualified I think. Clearly *some* attackers have deeper pockets. The ones with stuxnet-deep pockets though I think don't care about the average user. There also exists an interesting feedback loop between the drive-by exploiters and the victims in that the deeper pockets come about from having more victims -- compromise is indirectly the source of their revenue. So affecting exploitability affects the economics of the situation. For this reason, I don't think the deeper pockets will apply to the kinds of attacks average users are generally affected by. Time will tell, but for now we're still just seeing attacks on low-hanging fruit -- and the ROI for that low-hanging fruit is still extremely high (thanks to XP, no mandatory ASLR, etc). But I expect this to change

BTW thank you for your responses; even if we disagree, I think it's important to have these opinions represented (along with the reasons behind them, otherwise we'd be like that "I can prove it!" guy at your presentation ). To the people on twitter looking to this for "entertainment": I sincerely hope you all go away disappointed.

Also, thanks Matt for your post and clarification.

-Brad

[andreas]

I'll just try to address the issues I consider most relevant, otherwise we'll discuss forever.

It should be obvious that preventing a bug from happening is much more efficient at preventing exploitation than making life harder for the exploit author.

the problem with this in practice is that you either have to know where the bugs are or at least you have to know of all possible kinds of bugs to be able to develop prevention mechanisms for them (say, bounds checking against array bound violations, etc). now it so happens that noone knows how to do either (trust me, if you can demonstrate knowledge of either, you'll be very rich overnight). the best we have is that we know of some bug classes and have some usable prevention mechanisms against a subset of them.

The ability to completely eliminate all vulnerabilities and prove their non-existence in the context of commercial native software is (and will likely remain) a pipe dream. That’s not to say that attempts to work toward this are misguided – far from it – rather, it is important to understand the practical limitations of such an approach. Software vendors are at a disadvantage when it comes to eliminating vulnerabilities because they must find and eliminate all vulnerabilities whereas an attacker only needs to find one that is exploitable (a point which has been raised many times).

It is my central claim that you both are mistaken here (or maybe not, depending on the interpretation of "commercial native software"). It is entirely possible to address whole classes of vulnerabilities by addressing them on the right abstraction layer. Of course, fixing existing C/C++/ObjC software is indeed hard, and certainly not commercially viable for any substantial body of code. However, choosing an implementation language that doesn't expose raw memory, and proving soundness of compiler and runtime, is exactly the solution that provides absence of any memory corruption bugs.

In fact, it appears to me that the main difference between our points of view is that you folks say "C code can't be fixed, so we try to reduce risk", whereas I say "C code can't be fixed, so we better get rid of it in critical systems".

It is impossible to write bug-free software, I agree with that. However, it is possible to write software where whole classes of bugs are absent, just by designing your system in the right way.

[andreas]

Yes, I would like to get that straight, but unfortunately your view isn't supported by the evidence

I'm not sure there's much win in this line of argumentation, and i don't want to get too involved into a historic discussion of who invented what first. Back when I learned about ret2libc in the 90s, it was straight function calling. I'm aware of the chaining that happened, but the relevant point to me is: I now have powerful tools that automate so much of this process.

It's not a dead end if the PaX project isn't done yet. There are still lots of interesting things to do -- have you read pax-future.txt?

I've had a look, yes. Now imagine I find an arbitrary read vulnerability: how would all of this stop me from reading the private key from the process?

No, I was just curious if you had some evidence to support your claim that PaX technologies had only a limited effect.

The number of working exploits is above zero. That's all I need as evidence that the effect is limited.

Ok, but arguably the people working on CompCert are far above your average PHP developer in intelligence There are many reasons why people choose certain languages, even long after we know how bad they are. So I'm more interested in how expensive that Adobe Reader rewrite would be. For the idea to be seen through to its completion, all those average PHP developers need to be converted too -- we can't have the world relying on a small (or smaller) group of super-developers -- the cost would explode.

You're actually raising a very good point here that I haven't properly addressed yet (but of course thought about).

For starters, I'd hold that the CompCert people are not only way smarter than the average PHP developer, but also much smarter than the average PHP maintainer. PHP is such a mess, e.g. the fact that 0 == "0e12345678" is really surprising and has hit quite a few people off guard.

But the answer is: you need to have a small group of super-developers, and have them design a system. The key is to design the system in a way that allows programmers at the higher levels to get their job done efficiently, without requiring them to use supernatural abilities, and without risking those people to break your system. C as a language is totally out of the question for such an approach.

What I meant to imply here was that all ranges of companies have obvious self-interests that result in them making "short-cuts," especially when their customers are none-the-wiser. If annotations aren't provided to prove certain properties about the code, then those properties won't be validated, yes? So my interest wasn't in whether the proof does what it says, I understand that that's checked. I'm more concerned about what the actual proof says (what properties have been validated) and how we'd be able to know that information.

I have no good answer for the average joe here. A reasonably sized customer with a serious security interest will be able to read the proofs (or at least the propositions that are proven to hold).

Even if you're emulating though, you're retaining compatibility with some old, insecure code. Sure, now it can't feasibly compromise your entire system (we hope) but an owned browser is nearly as bad these days. So I'm not concerned with covert channels, but with how you prevent exploitation of legacy code without something like the techniques in PaX.

No question here. To be really secure, you have to keep legacy systems at arm's length on another host. And of course legacy systems will do better with legacy mitigation techniques.

Well, that should be qualified I think. Clearly *some* attackers have deeper pockets. The ones with stuxnet-deep pockets though I think don't care about the average user.

I don't think I care about the average user, at least not initially. They don't care enough about security to give up convenience. But there's the fact that you cannot buy any reasonable defense, even with stuxnet-deep pockets, and even if your job includes defending power plants or countries.

BTW thank you for your responses; even if we disagree, I think it's important to have these opinions represented (along with the reasons behind them, otherwise we'd be like that "I can prove it!" guy at your presentation ). To the people on twitter looking to this for "entertainment": I sincerely hope you all go away disappointed.

Well, i do hope that there are people who consider a meaningful discussion better entertainment than pointless flaming. And I certainly appreciate your feedback.

-Andreas

[mmiller]

It is my central claim that you both are mistaken here (or maybe not, depending on the interpretation of "commercial native software"). It is entirely possible to address whole classes of vulnerabilities by addressing them on the right abstraction layer. Of course, fixing existing C/C++/ObjC software is indeed hard, and certainly not commercially viable for any substantial body of code. However, choosing an implementation language that doesn't expose raw memory, and proving soundness of compiler and runtime, is exactly the solution that provides absence of any memory corruption bugs.

In fact, it appears to me that the main difference between our points of view is that you folks say "C code can't be fixed, so we try to reduce risk", whereas I say "C code can't be fixed, so we better get rid of it in critical systems".

I define commercial native software with a bias toward Windows and the applications that run on it. The problem with arguing for rewriting native code in a managed language is that it's a hindsight argument akin to "if we knew what we knew today we could have avoided the problems that we have now". In practice rewriting native software carries a substantial cost that software vendors are unlikely to bite off for the exclusive reason of improving the security of their products. Some of the costs of a rewrite are not obvious; for example, rewriting legacy software means retaining legacy behavior and achieving performance that is on par with the previous implementation. Then there's the obvious cost associated with porting the application to the new language and runtime while retaining the intended contracts and assertions that were defined by developers who, in all likelihood, are no longer at the company. This is all assuming that a rewrite is possible, of course. In some application domains it is not possible to write managed software. The best example of this today, at least on Windows, is the kernel and the device drivers loaded by the kernel (although there are research projects working toward this).

Fundamentally I agree that rewriting native software in a managed language can eliminate memory safety vulnerabilities (assuming the runtime and the native code it builds on, such as the OS, have been proven safe -- easier said than done today). However, as I described above, I believe the cost associated with doing this in the context of most commercial software vendors makes doing so impractical. The core business of software vendors is to develop and improve the functionality of their products, so unless a rewrite is aligned with the core business interests, it is quite unlikely to expect an investment in doing so even if the vendor has a strong security tenet. As we look toward the future I'm inclined to believe that an increasing amount of software will be written in managed code. That said, I really don't to see this shift make a dent in the dominance of native code for quite some time (at least in certain application domains).

I've had a look, yes. Now imagine I find an arbitrary read vulnerability: how would all of this stop me from reading the private key from the process?

What if there are no determinsitic mappings containing useful data in the address space of the process (and the attacker cannot coerce the program to create any, such as by heap spraying)? In these cases an arbitrary read (or write) becomes much less valuable. I'm not claiming this is often the case today, I'm just merely pointing out that a complete implementation of ASLR can mitigate arbitrary read and arbitrary write vulnerabilities in the common case.

The number of working exploits is above zero. That's all I need as evidence that the effect is limited.

I don't think this conclusion follows from the evidence you cited. Effectiveness in this case needs to be judged with respect to the number of exploits that are able to bypass mitigations vs. the total number of vulnerabilities that could potentially be exploited. Saying that mitigations have limited effect because 3 / 100 exploits (numbers picked out of the air) were able to reliably bypass mitigations does not illustrate limited effect -- in fact, given such numbers, I would argue in favor of the opposite position. Imperfection does not imply that something is ineffective in practice.

Matt

[PaX Team]

The workaround for correctly implemented ALSR is to find an information disclosure bug that allows reconstruction of the base address for whatever you then turn into gadgets.

finding the base address is enough only if you know what exact binary that base address belongs to (using gadgets requires byte precision), otherwise you'll also have to figure out what's in the binaries as well. this is easy if your target uses widely distributed precompiled binaries, not so easy otherwise (you will need control over the address of the leak, that is usually not the case for these kinds of bugs).

Essentially, a combination of a write memory corruption and a read memory corruption is what's needed to beat all the combined technologies. As a thought experiment, consider perfect bugs, which would allow me to read from and write to arbitrary memory addresses, an arbitrary number of times. Do you really think there is any conceivable mitigation that could not be beaten?

the reason i asked right at the beginning whether you were familiar with the PaX docs or not is because you'll find the answer to your exact question above right in the main doc . let me quote myself:

The goal of the PaX project is to research various defense mechanisms
against the exploitation of software bugs that give an attacker arbitrary
read/write access to the attacked task's address space.

so the whole project is based on the very threat model you've just described. the answer to your question is that "it depends". in particular it depends on what you consider "beaten". if you're a remote attacker (i.e., you don't have a priori rights to execute arbitrary code on the target system) who wants to execute his own code on the target - it can be made impossible. if you want to do something useful by executing existing code then it depends on whether the attacked program (including all the libraries it maps) can a priori do that something or not (assuming there's some form of control flow integrity in there already). typically, exploiting kernel bugs is not in their repertoire. suddenly all those sandboxes are beginning to make a whole lot more sense .

I do not claim that PaX is useless, far from it. It makes life for the attacker harder. But I do claim that it it represents an evolutionary dead end in terms of computer security. It only addresses memory corruption issues, and it only makes it harder to exploit those, not impossible.

the fundamental problem everyone in this field is trying to address is the existence of exploitable bugs. there are two ways to solve the problem: either get rid of the exploitable bugs (by not creating them in the first place or by finding them) or make them not exploitable. the reason we're doing the latter and not the former is very simple: we don't have the resources for the former. never mind the fact that we also think that it is not possible to eliminate exploitable bugs (we don't know if there is a fundamental reason for that, but the hard evidence from real life shows it to be the case). why we focus on memory corruption bugs is simple as well: if we cannot guarantee protection from this class of bugs, there is no way we can guarantee protection for anything else (any protection mechanism for any kind of bug is based on the assumption of memory safety), one has to start with them before one can address other kinds of bugs. what we achieved so far shows that in fact you can make certain very real life memory corruption bugs not exploitable for anything useful (DoS is not typically the objective for an attacker, just ask any drive-by exploit kit buyer). and we do that not only for userland but the kernel as well. i think you can call this approach many things but a dead end .

We're talking about:

1. Throw away all old code

your approach failed already right at this point. it is simply never going to happen. i thought Java taught everyone that lesson already .

2. Choose an implementation language that doesn't allow the programmer to write code that will lead to memory corruption

second failure: for all the talk about type safe languages and whatnot, i have never in my life seen even a simple thing like an sshd implemented in any of those change-the-world languages. and we're not talking about proof of concepts here but a drop-in replacement for openssh. you could have already made a business out of such a thing, yet all these decades in academic and industrial research haven't managed to produce anything like that. and you seriously think that suddenly a million coders will begin to rewrite the universe?

3. Prove correctness of the language implementation and its runtime.

this is something we have yet to see as the fundamental problem with proofs is that of scalability (or lack thereof). but i'm all for progress here since results in this field can be applied to other languages as well.

4. Write new code for reader

i can see the Adobe board meeting in 10 years, it will go something like this:

boss, we have this new thing that would allow us to get rid of all the security bulletins in the future.
oh cool, what's the cost/benefit analyst say?
err, never mind!

The reader itself is probably quite a lot more code, but I do not even need to consider it in my calculation, since memory corruption issues are dealt with in the compiler and runtime. The key is to provide the programmer with an abstraction where he can't corrupt memory. So essentially, the cost is rewriting the application, plus some cost amortized through all users of that language.

i can't help but notice that for all the 'talking down' on 'only' trying to address memory corruption bugs in PaX you don't seem to claim more than the very same thing for these projects .

First of all, I'm not just concerned with normal users. Everybody is affected by the current disaster. Eventually, there will be no alternative to the more secure technologies trickling down to the normal user. Then, I do think that it is possible to completely eliminate whole classes of vulnerabilities. The fact that it hasn't been done yet only shows the limits of the attempts used in the past, not that it is impossible. Nobody would have thought it was feasible to prove the correctness of an optimizing C compiler either, until someone came and did it.

i'm not sure what a correct C compiler helps here (i can count on one hand the exploitable bugs introduced by a compiler) when your future secure system won't even be written in C, but have you seen what their correct C compiler can do? it doesn't even support C99. nor does it produce code comparable to gcc or other good compilers (no, gcc-3 is not a good basis for comparison). as i said above, the basic problem in this whole proof business is that of scalability (will they ever be able to prove correctness of all the gcc (or clang/llvm) optimization passes?).

Transitioning between generations of systems has always happened. There are two ways of transition: keeping legacy hosts in their own isolated networks with data flow enforced by more secure machines, or running legacy code in emulators. I'm a bit careful with the latter solution in high security environments, cache effects could establish hidden channels from the vulnerable legacy code into the more secure systems.

you forget the fundamental motivation for any kind of transition: it must be worth it. and you can guess what users will choose between a supposedly secure system with no useful apps and/or slow ones vs. what they have right now... (on a smaller scale Vista taught that lesson to Microsoft).

[PaX Team]

It is my central claim that you both are mistaken here (or maybe not, depending on the interpretation of "commercial native software"). It is entirely possible to address whole classes of vulnerabilities by addressing them on the right abstraction layer.

i think you just misunderstood me. i wasn't talking about some classes of bugs in the part you quoted, i was talking about all (exploitable) bugs, not just a subset of them. and in that light it seems you agree with me and admit that it's possible to solve only part of the problem with your approach. we do differ on what the 'best' solution is though since we have different constraints (and assumptions) to work under. for you it seems entirely feasible to rewrite everything in a new language, for me it's a pipedream that many have tried and failed at (for a reality check you can try floating the idea on lkml and see what you get ). but let time be the judge.

In fact, it appears to me that the main difference between our points of view is that you folks say "C code can't be fixed, so we try to reduce risk", whereas I say "C code can't be fixed, so we better get rid of it in critical systems".

yes that's correct with the addition that in my experience it's not an either/or difference but rather a different stage of evolution. the 'all our problems are due to X, let's redo everything in Y' approach is a very natural first thought to anyone encountering this problem space but reality checks in eventually when one is faced with the sheer amount of resources needed to pull it off. and while we're waiting for the perfect we've got to use something every day and settle for the good enough instead .

It is impossible to write bug-free software, I agree with that. However, it is possible to write software where whole classes of bugs are absent, just by designing your system in the right way.

out of curiosity, how do you imagine your safe language to allow (presumably safe) runtime code generation? these days users expect a JIT Javascript compiler in their browser...

[thomaspollet]

theoretically, formal models could be used to approximate the halting problem by proving partial correctness for current codebases

• if bugclasses can be formalized, declarative pattern matching to locate existing bugs could be more effective (cfr. smpl/coccinelle , I think coverity also uses complex semantic analysis to locate bugs [?] )
• when reviewing code fragments, a theorem prover could be used to enumerate and validate all constraints that would prevent memory corruption for some statement, thus greatly helping the auditor.