encrypted executables

[ari edelkind]

des at des.no wrote:
> ari edelkind <edelkind-freebsd-hackers at episec.com> writes:
> > Keep in mind that ptrace(PT_ATTACH,...) will fail if a process is
> > already being traced.  As for core files, a process can use
> > setrlimit(RLIMIT_CORE,...) to disable core dumps, and individual memory
> > pages may be encrypted or unloaded, to be decrypted or loaded on
> > demand.
> 
> The person running the application can trivially replace ktrace(),
> ptrace() and setrlimit() with non-functional stubs using LD_PRELOAD.

And any application that executes its own code before running the
system's dynamic loader -- or is statically linked, for that matter --
is free to unset LD_PRELOAD.

There are many attack vectors.  There are plenty of countermeasures.
There are numerous attacks on each countermeasure.  It goes on.  This is
all common knowledge, even among those creating anti-reverse-engineering
techniques; in fact, it's usually prominently stated in an included
disclaimer.

It's unfortunate to note that, in many countries these days, the most
effective deterrent against attacks on binary encryption is legal
action.  Some corporations add just-in-time page decryption to their
binaries specifically for this recourse (e.g., against a competitor who
creates applications that hook into the original software).

ari