I can tell you one problem right away. This part won't work with pipe streams like php files:

    //get the length of file in bytes
        fseek(file, 0, SEEK_END);
        byte_length = ftell(file);
        fseek(file, 0, SEEK_SET);

To get the load() function working correctly 100% of the time, one usually needs to read from the stream progressively and use realloc() when needed.

DinoCoderSaurus gave some advice about it here: http://cs50.stackexchange.com/questions/15848/pset6-webserver-hello-php-not-loading-properly
***
**EDIT to tackle index.php problem:**

Look what's happening here:  

    result = (access(linkphp, F_OK) != -1) ? linkphp : NULL;
    if (result != linkphp)
        free(linkphp);

    result = (access(linkhtml, F_OK) != -1) ? linkhtml : NULL;
    if (result != linkhtml)
        free(linkhtml);

    return result;

Now suppose there is a file called index.php inside the directory. That first part of the code will make the variable `result` point to `linkphp`, **but you don't return it**, so the computer proceeds to analyze the rest of the code.

What comes next? Well, it checks for the existence of `index.html`... and let's suppose it doesn't exist. `result` will now point to `NULL`.

At the end of the function, `NULL` will be returned, even though the correct answer would be to return `linkphp`. See the problem there?