0

I wanted to try a different approach to PSET4, but not sure if it's stupid or not. It seems to be 'kind of' working so hoping to get someone with more knowledge to help me understand whether this is a path worth pursuing or not.

I'm attempting to read through the entire file and get the memory addresses of the 'JPGHeaders' and store them in an array. My plan after that will be to do some memory address math to find the number of bytes to write and then write the correct number of bytes from each address until the end of the file, looping over that array.

For some reason, matches are getting stored as 'nil' in my array; I thought at first it may be because random bytes within the JPG had that same combo, but when I changed it to jump 512 bytes it does the same thing.

Is this approached incorrect from the beginning, or is there something I don't understand about FILE I/O or matching that's making it incorrect?

Code snippet in question:

    FILE* ptr = fopen("card.raw", "r"); // open the file; point to the head to be used to find addresses from the beginning of the file
FILE* matched_address; // create a pointer in the file when an address matches

char *infile = "card.raw";
FILE* readptr = fopen(infile, "r"); // create a pointer to read from the file;

if (readptr == NULL){
    printf("Could not open %s.\n", infile);
    return 2;
    }

FILE* locarray[50]; // initialize an array of memory addresses
int read_index=0, i=0; // will store the addresses based how many 'jpgHead' sized chunks they are away from the beginning of the file

JPGHead jpgMatch;

char s[sizeof(jpgMatch)]; // create an array to store 4 byte sized elements to test if they match a jpg setting; this will be overwritten

// Now for the action: we read through the file 4 bytes at a time, finding the addresses of those that match the JPG signature, and storing those addresses to an array...
while(fgets(s, sizeof(jpgMatch), readptr) != NULL){
    if (s[0] == (char)0xff && s[1] == (char)0xd8 && s[2] == (char)0xff && (s[3] <= (char)0xef || s[3] >= (char)0xe0) )
    {
        matched_address  = ptr + (read_index *  sizeof(jpgMatch)); // the address will just be the index * the size of chunks we're moving forward away from the head of the file
        locarray[i] = matched_address; // store the address in an array; only increase the arrays index when you've matched;
        i++;
    }
    read_index++; //  
}

for (int j = 0; j < 50; j++)
    {
        printf("array element %i is %p \n", j, locarray[j]);
    }

return 0;
}

EDIT: Here's the results of stored arrays. Many look like memory addresses but not sure why there are 'nils': array element 0 is 0x7f220959ff48

array element 1 is 0x7f2209388816

array element 2 is (nil)

array element 3 is 0x7ffdb3646540

array element 4 is 0x7f2208cb6d28

array element 5 is 0x7ffdb3646650

array element 6 is 0x7f2208cc31a8

array element 7 is 0x3d8f538

array element 8 is 0x7ffdb3646640

array element 9 is 0x7f22095a2ae8

array element 10 is (nil)

array element 11 is 0x7f220959ff98

array element 12 is 0x7f220959fa10

array element 13 is 0x400411

array element 14 is 0x7f2208cc3d78

array element 15 is 0x400300

array element 16 is 0x100000000

array element 17 is 0x100000830

array element 18 is (nil)

array element 19 is 0x7ffdb3646708

array element 20 is 0x7ffdb36466e0

array element 21 is 0x7f220959ff98

array element 22 is 0x1

array element 23 is 0x7f22095a2520

array element 24 is 0x7f22095a21c8

array element 25 is 0x7f2209388991

array element 26 is (nil)

array element 27 is 0x7f220959ff98

array element 28 is 0x1

array element 29 is (nil)

array element 30 is 0x7ffd00000001

array element 31 is 0x7f22095a21c8

array element 32 is (nil)

array element 33 is (nil)

array element 34 is (nil)

array element 35 is (nil)

array element 36 is (nil)

array element 37 is 0x7f22095a2520

array element 38 is 0x7ffdb3646650

array element 39 is 0x7ffdb3646640

array element 40 is 0xf63d4e2e

array element 41 is 0x400411

array element 42 is 0xffffffff

array element 43 is 0x7ffdb36467a8

array element 44 is 0x7f2208cc31a8

array element 45 is 0x7f220959fa10

array element 46 is 0x7f22095a21c8

array element 47 is (nil)

array element 48 is 0x1

array element 49 is 0x4008ad

Improve this question

1 Answer 1

Reset to default
1

I didn't to a thorough analysis, but I did see a few things and have some thoughts. First, the signature test will pass any value for the 4th byte. Because of the or clause, (s[3] <= (char)0xef || s[3] >= (char)0xe0) it's like saying x <= 5 OR x >= 2, which is any number.

After that, are you sure that it is finding signatures? Maybe a printf in the if statement's code block is in order. I don't know because I would want to test the entire program without guessing what the rest is.

It's an interesting concept and an interesting exercise, but not efficient as a practical matter. The problem is file I/O efficiency.

File I/O is slower than memory I/O. If it's a solid state drive (SSD), it's a little worse. If it's traditional spinning hard disks, the file I/O can be 1000 or more times slower to execute because of overhead. It takes a lot more time to position the drive heads and to rotate to the right portion of the track to be read, plus other system overhead to do a read or a write. In fact, to read up to the size of a disk data block (typically 4k, system dependent), the overhead will be about 95% of the total read/write time.

How does this relate to this program? Your concept essentially requires the file to be read at least twice. Additiionally, this program assumes 50 files to be recovered. What if that were not an accurate number? If less, it shouldn't be a problem. If it's more, oops! It would be far more efficient to process the data by linearly reading through the input file, and when a signature is detected, close the output file, if any, and open the next, and just keep going to the end.

BTW, check50 uses a different card file with a different number of files. I'm not saying if it's more or less though. ;-)

If this answers your question, please click on the check mark to accept. Let's keep up on forum maintenance. ;-)

Improve this answer
5
  • Got it, but from a performance perspective, doesn't this use a lot less memory + also allow for easier writing (by just subtracting memory addresses)? Also wouldn't it work better for files that for example aren't contiguous on the disk, but we knew roughly what size they were? Sorry to bombard with questions, just trying to understand the ideas here. Also you don't have to have the memory allocated (I know it's small) but if the files were much bigger couldn't that affect performance on a memory limited computer? Pretty dumb mistake on the logical operator though...
    – borker
    Jul 7, 2016 at 0:02
  • All good questions. Would your method use a lot less memory? Depends on how much yours is using, but since you are reading 4 bytes at a time, then yes, but at tremendous cost. Reading 4 bytes at a time means 256 reads per kb vs. 2 reads if you read in a 512 byte block at a time. Multiply that by a 20 MB file and you have 5 million reads and another 5 million writes vs. 40k reads and writes. The run time goes from a few seconds to many minutes. (continued.....)
    – Cliff B
    Jul 7, 2016 at 1:14
  • By today's standards, 512 bytes is a small amount of memory. Your computer probably has several gigabytes of memory. In a production system with a database application, they may well load whole tables or databases of multiple gigabytes, or even terabytes or more into memory purely to maximize performance.
    – Cliff B
    Jul 7, 2016 at 1:16
  • "Also wouldn't it work better for files that for example aren't contiguous on the disk, but we knew roughly what size they were?" TAn entirely different question that depends on program design, but I'd design it to take in as large a data block as I could to analyze and process it, both to minimize read overhead and to get a bigger picture of what's going on with the corrupted data. The basic rule of thumb is to try to minimize I/O on peripherals and maximize memory usage. CPU activities are the fastest with the least overhead, followed by memory operations, and finally peripheral ops.
    – Cliff B
    Jul 7, 2016 at 1:22
  • Cliff, thanks for all of the comments, here's the array; looks to me like some (but not all) memory addresses are being stored... but the approach makes sense. So really I should just scrap this code and write it as suggested in the walkthrough?
    – borker
    Jul 7, 2016 at 15:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .