pset5 Can we remove duplicate words from text?

Question

In pset5, we are trying to minimise the time taken to check for misspelled words. Therefore, I'm thinking it would be a good plan to remove duplicate words from the file text, before checking to see if each word occurring in text, is in the dictionary. Am I right about removing duplicate words first up? I am perplexed, because the pset5 specification seems not to allow for us to weed out the duplicate words from text.

I am hesitant to plunge into the coding tasks, without having some advice on this point. Pretty sure that I'm missing something here, as I'm among the "less comfortable".

Answer 1

For doing such a thing you have to edit speller.c and the specifications doesn't let you do it.

By the way, an important note is that by doing that, you don't gain any improvement.

It's not always a bad thing, but if you have only to spell-check the file (as in this case) you double your work. I wrote "double" because you have to do the same process. You have to:

• load every single word in a data structure of "existing word"
• check for each word if it already exist in the data structure of "existing word";
• create a new data structure and load the dictionary in it;
• check for each word of the file if it exist in the data structure "dictionary"

It's not efficient. The same thing, two times.

Doing some math I think you quite double the time of execution.