Your code has at least two problems, probably more. First, your
isjpgheader() has to check the first 4 bytes of header. Now look at how it's defined:
BYTE header; The 4th byte doesn't exist. Only 3 bytes, byte 0, byte 1, and byte 2 exist.
Next, you have an infinite loop in your first while loop. It depends on finding that first signature, but since it isn't finding it, it loops forever. It would be a good idea to also check for EOF to terminate an infinite read loop. The reason for the infinite loop is that you read the first 4 bytes, then you reset the pointer back 4 bytes. You're reading the same 4 bytes over and over!
That will get you going, but you might want to reconsider your design. The code is far more complicated than it needs to be! There are 6 fread calls, multiple variables to hold the same data, multiple fwrite and fseek calls, etc. Over time, I've noticed that the more fread calls, the more error-prone the code becomes. You should be thinking about efficient code.
There's an old programming adage, "KISS - Keep It Simple, Stupid!" I tell myself this on just about every project that I work on. Also, if it feels too complicated, it probably is. This program can be implemented with a single fread, a single fwrite, and no calls to fseek. All you need to do is read in a block, decide what to do based on the first 4 bytes and whether an output file is open, and then do it.
Just some things to think about.
IF this answers your question, please click on the check mark to accept. Let's keep up on forum housekeeping. ;-)