Port Win32 DLL hook to Linux

I have a program (NWShader) which hooks into a second program's OpenGL calls (NWN) to do post-processing effects and whatnot.

NWShader was originally built for Windows, generally modern versions (win32), and uses both DLL exports (to get Windows to load it and grab some OpenGL functions) and Detours (to hook into other functions). I'm using the trick where Win will look in the current directory for any DLLs before checking the sysdir, so it loads mine. I have on DLL that redirects with this method:

#pragma comment(linker, "/export:oldFunc=nwshader.newFunc)

To send them to a different named function in my own DLL. I then do any processing and call the original function from the system DLL.

I need to port NWShader to Linux (NWN exists in both flavors). As far as I can tell, what I need to make is a shared library (.so file). If this is preloaded before the NWN executable (I found a shell 开发者_运维技巧script to handle this), my functions will be called. The only problem is I need to call the original function (I would use various DLL dynamic loading methods for this, I think) and need to be able to do Detour-like hooking of internal functions.

At the moment I'm building on Ubuntu 9.10 x64 (with the 32-bit compiler flags). I haven't been able to find much on Google to help with this, but I don't know exactly what the *nix community refers to it as. I can code C++, but I'm more used to Windows. Being OpenGL, the only part the needs modified to be compatible with Linux is the hooking code and the calls. Is there a simple and easy way to do this, or will it involve recreating Detours and dynamically loading the original function addresses?

---

The library preloading can be done through LD_PRELOAD. From there you want to look at the dlopen and dlsym calls to get at the actual functions in the original library. This is all if you want to do things by hand.

You can also look at modifying ltrace in a way such that you provide the functions to hook (via the -e flag) and let ltrace handle the bookkeeping for you.

[Edit] An example of doing this by hand:

#include <dlfcn.h>
#include <stdio.h>

int (*orig_puts)(const char *);

int puts (const char * str) {
    void * handle = dlopen("/lib/libc.so.6", RTLD_NOW | RTLD_GLOBAL);
    orig_puts = dlsym(handle,"puts");
    fprintf (stderr,"I have hooked your puts\n");
    return orig_puts(str);
}

And with a program such as

#include <stdio.h>

int main () {
    puts ("Hello World");
    return 0;
}

You get the following:

$ ./a.out
Hello World
$ LD_PRELOAD=./libhook ./a.out
I have hooked your puts
Hello World

---

This sounds like what you are looking for. You may have found a solution already, but I thought I would pass this on. I use linux and play NWN and would love to be able to use nwshader. OGC (referred to in the article) seems like it is some kind of multiplayer cheat that works by interrupting Opengl, much like what nwshader does but for a different purpose.

http://aimbots.net/tutorials/14575-detours-linux-windows.html

Detours for Linux & Windows

This is a basic "Hello world" detour example in C++.
It does not make use of the Microsoft detour library.
Therefore it works on Windows, Linux and Mac.

I used the detour and undetour functions from OGC, but corrected it for IA64, and I also corrected the bug that made it crash on Fedora.
Also, it works with C++. If you want to use it with pure C, you need to remove the C++ style typecasts, as well as the template.
You don't need the template in C anyway, since C lets you convert any pointer to void* without any error or even warning.
Works with IA-32 & IA-64 & AMD64 x86 processors.

To be fully working, you would need to include a disassembler and adjust relative jumps in the 5+ bytes detourlength. You would also need to take care if you are writing over to the next memory page. (It almost never happens, but it could happen.)

On IA-64, you can maximally jump 4 Gigabytes. That should be sufficient for any normal program, however.

#if ( defined (_WIN32) || defined (_WIN64) )
    #define WIN32_LEAN_AND_MEAN
    #define WIN64_LEAN_AND_MEAN
    #include <windows.h>

    #define unprotect(addr,len) (VirtualProtect(addr,len,PAGE_EXECUTE_READWRITE,&oldprot))
    #define GETPAGESIZE()        getpagesize()

    DWORD oldprot ;

    unsigned long getpagesize (void)
    {
        static long g_pagesize = 0 ;
        if (! g_pagesize)
        {
            SYSTEM_INFO system_info ;
            GetSystemInfo(&system_info) ;
            g_pagesize = system_info.dwPageSize ;
        }
        return (unsigned long) g_pagesize ;
    }

    #define CLEAR_SCREEN "cls"

#else // LINUX / UNIX / OS X
    #include <unistd.h>
    #include <sys/mman.h>

    #define unprotect(addr,len)  (mprotect(addr,len,PROT_READ|PROT_WRITE|PROT_EXEC))
    #define GETPAGESIZE()         sysconf (_SC_PAGE_SIZE)
    #define CLEAR_SCREEN "reset"
#endif


#include <iostream>
#include <cstdlib>
#include <cstring>


    unsigned long uslngPageSize = 0    ;
    unsigned long uslngPageMask = 0    ;








    #define JMP_OPCODE 0xE9
    #define OPCODE_LENGTH 1
    #define DATATYPE_ADDRESS int
    #define ADDRESS_LENGTH (sizeof(DATATYPE_ADDRESS))
    #define MIN_REQUIRED_FOR_DETOUR (OPCODE_LENGTH + ADDRESS_LENGTH)
    #define INT_DETOUR_FACTOR 1
    #define OPCODE_NOT_DEFINED 0



// offset[ENGINE][FUNCTION_NAME] ;
// detourlength[ENGINE][FUNCTION_NAME]

#define HOTPATCH(FUNCTION_NAME) \
    original_##FUNCTION_NAME = TemplateFuncInterceptFunction( \
                                                             original_##FUNCTION_NAME, \
                                                             reinterpret_cast<unsigned long> (&FUNCTION_NAME), \
                                                             reinterpret_cast<unsigned long> (&modified_##FUNCTION_NAME), \
                                                             static_cast<unsigned long> (FUNCTION_NAME##_COPY) \
                                                            )

#define UNPATCH(FUNCTION_NAME) \
    unpatchfunc( reinterpret_cast<void*>(reinterpret_cast<unsigned long>(&FUNCTION_NAME)), reinterpret_cast<unsigned char*> (reinterpret_cast<unsigned long>( original_##FUNCTION_NAME)), static_cast<unsigned long> (FUNCTION_NAME##_COPY))



#define NATURALIZE(FUNCTION_NAME) \
    Naturalized_##FUNCTION_NAME = FuncConvertAddress(Naturalized_##FUNCTION_NAME, &FUNCTION_NAME)


template <class DataType>
DataType FuncConvertAddress(const DataType dt_FunctionPointer, unsigned long uslng_FunctionAddress)
{
    return reinterpret_cast<DataType> (uslng_FunctionAddress) ;
}




void* FuncGetPage(const unsigned long &uslngVirtualMemoryAddress)
{
    return reinterpret_cast<void*> (uslngVirtualMemoryAddress & uslngPageMask) ;
}


void* InterceptFunction (void* voidptr_AddressOfDetouredFunction, unsigned long uslng_CopyLength, void* voidptr_AddressOfDetourFunction)
{
    DATATYPE_ADDRESS Relocation ;
    //printf("copy length: %ld\n", uslng_CopyLength);
    //printf("MIN_REQUIRED_FOR_DETOUR : %d\n", MIN_REQUIRED_FOR_DETOUR );
    void* voidptr_BackupForOriginalFunction = malloc( uslng_CopyLength + MIN_REQUIRED_FOR_DETOUR ) ;
    //printf("Sizeof Backuppointer %ld\n", sizeof(voidptr_BackupForOriginalFunction));
    //printf("Sizeof AddrDetouredFunction %d\n", sizeof(voidptr_AddressOfDetouredFunction));

    memcpy( voidptr_BackupForOriginalFunction, voidptr_AddressOfDetouredFunction, uslng_CopyLength) ;

    if (OPCODE_NOT_DEFINED)
    {
        printf("Error: OP-Code not defined\n.") ;
        exit(EXIT_FAILURE) ;
    }

    memset( reinterpret_cast<void*> (reinterpret_cast<unsigned long> (voidptr_BackupForOriginalFunction) + uslng_CopyLength),
            JMP_OPCODE, OPCODE_LENGTH ) ;



    Relocation = static_cast<DATATYPE_ADDRESS> (reinterpret_cast<unsigned long> (voidptr_AddressOfDetouredFunction)
                  - (reinterpret_cast<unsigned long> (voidptr_BackupForOriginalFunction)
                  + MIN_REQUIRED_FOR_DETOUR)) ;


    memcpy( reinterpret_cast<void*> ( reinterpret_cast<unsigned long> (voidptr_BackupForOriginalFunction)
             + uslng_CopyLength + OPCODE_LENGTH), &Relocation, ADDRESS_LENGTH) ;



    unprotect(FuncGetPage(reinterpret_cast <unsigned long> (voidptr_AddressOfDetouredFunction)),uslngPageSize) ;

    memset(voidptr_AddressOfDetouredFunction, JMP_OPCODE, OPCODE_LENGTH) ;

    Relocation = static_cast<DATATYPE_ADDRESS> ( reinterpret_cast<unsigned long> (voidptr_AddressOfDetourFunction)
                  - (reinterpret_cast<unsigned long> (voidptr_AddressOfDetouredFunction)
                  + MIN_REQUIRED_FOR_DETOUR)) ;

    memcpy( reinterpret_cast<void*> (reinterpret_cast<unsigned long> (voidptr_AddressOfDetouredFunction)
             + OPCODE_LENGTH), &Relocation, ADDRESS_LENGTH) ;
    unprotect(FuncGetPage(reinterpret_cast <unsigned long> (voidptr_BackupForOriginalFunction)),uslngPageSize) ;

    return voidptr_BackupForOriginalFunction ;
}


// C++ is typesafe, they said...
// I say: Yes, but at which price ?
template <class DataType>
DataType TemplateFuncInterceptFunction( DataType dt_Original_Function, unsigned long uslng_FunctionAddress,
                                        unsigned long uslng_modified_FunctionName, unsigned long uslng_DetourLength)
{
    return reinterpret_cast<DataType>
            ( reinterpret_cast<unsigned long>
                ( InterceptFunction( reinterpret_cast<void*> (uslng_FunctionAddress),
                                     uslng_DetourLength,
                                     reinterpret_cast<void*> (uslng_modified_FunctionName)
                                   )
                )
            );
}


void SayHello()
{
    printf("Hello World\n");
}


void modified_SayHello()
{
    printf("**** World\n");
}

void (*original_SayHello)();
//#define SayHello_COPY 9
#define SayHello_COPY 6






void unpatchfunc(void* patched_function, unsigned char* original_function, unsigned long uslng_DetourLength)
{
    //DWORD dw_OldProtect;
    //VirtualProtect(patched_function, uslng_DetourLength, PAGE_EXECUTE_READWRITE, &dw_OldProtect);
    unprotect(FuncGetPage(reinterpret_cast<unsigned long>(patched_function) ), uslngPageSize) ;
    unsigned int intIndex;
    for( intIndex = 0; intIndex < uslng_DetourLength; ++intIndex)
        *( (unsigned char*) patched_function + intIndex) = *(original_function + intIndex) ;

    //VirtualProtect(patched_function, uslng_DetourLength, dw_OldProtect, &dw_OldProtect);
    unprotect(FuncGetPage(reinterpret_cast<unsigned long>(patched_function) ), uslngPageSize) ;
    if(original_function!=NULL)
        free( (void*) original_function) ;
}


int main()
{
    system( CLEAR_SCREEN ) ;
    uslngPageSize = GETPAGESIZE() ;
    uslngPageMask = ( ~(uslngPageSize - 1) ) ;
    printf("PageSize: %ld\n", uslngPageSize) ;
    printf("PageMask: 0x%08lX\n", uslngPageMask) ;
    SayHello() ;
    printf("Hotpatching now!!!\n") ;
    HOTPATCH(SayHello) ;
    printf("Hotpatched:\n") ;
    SayHello() ;
    printf("Backup:\n") ;
    original_SayHello() ;
    printf("Unpatching now\n") ;
    UNPATCH(SayHello);
    // expands to: unpatchfunc( (void*) SayHello, (unsigned char*) original_SayHello, (int) SayHello_COPY) ;
    printf("Unpatched:\n") ;
    SayHello() ;
    printf("EXIT_SUCCESS\n") ;
    return EXIT_SUCCESS ;
}

Edit: Note that if you include this function in a 64 bit shared library/dll on Linux, you will get a segmentation fault. This is because 64-Bit shared libraries can only be compiled with -fPIC, which makes detouring more difficult, because you have to read the PLT before each jump. You need to compile the shared library as 32 bit Shared Object (-m32) and run it with a 32-Bit executable.

---

Write your own oldfunc in a shared library and preload it, as you've mentioned. But also write some initialization that calls dlopen() on the original library, and dlsym() to get the function pointer to the original oldfunc. (These functions are the Unix equivalents of LoadLibrary and GetProcAddress respectively.)

---

You may look at a method of functions redirection in shared ELF libraries. There's code attached. It allows you to hook a particular function from a particular module.