Pset 6 Words are randomly not being added to the dictionary

Question

For the most part my solution to pset6 works, it correctly fetches the tweets, graphs them, and analyzes them, but for some reason some words will not get added to my dictionary. For example hate returns :| while hated returns :( . I don't know why words right next to each other wouldn't be added to the dictionary. I think my code is right but I am not sure if this issue stems from me not knowing some fact about lists. If any one know what this might be please help me...

Answer 1

I think your list usage is fine, but a see a few possible problem spots.

1) In your init function you open your positives and negatives documents with "r" which already means you are reading the file, but then for each line you call f.readline() which maybe would skip every other line.

2) In your analyze function you seem to be trying to convert the whole text to lowercase, but from what I've read that doesn't work. I may be wrong on this though, but you may find these two threads helpful, since I think the idea is to change each word to lowercase when you check. This probably isn't a problem now, but may become a problem later on.

https://stackoverflow.com/questions/14067267/lower-case-from-a-text-file

https://stackoverflow.com/questions/38908966/change-all-text-in-file-to-lowercase

On a side note, are printing self.positive and self.negative in both functions to debug? If you are thats ok, but otherwise I don't think you need to do that.

Answer 2

from nltk.tokenize import TweetTokenizer import sys

class Analyzer(): """Implements sentiment analysis."""

def __init__(self, positives, negatives):
    """Initialize Analyzer."""
    # TODO
    #__init__ loads positive and negative words into memory in such a way that analyze can access them.
    self.positive = []
    self.negative = []
    with open(positives, "r") as f:
        for line in f:
            if f.readline().startswith(';') == False:
                self.positive.append(line.strip())
    print(self.positive)
    with open(negatives, "r") as f:
        for line in f:
            if f.readline().startswith(';') == False:
                self.negative.append(line.strip())
                #print(line.strip())
    print(self.negative)

def analyze(self, text):
    """Analyze text for sentiment, returning its score."""

    # TODO
    #analyze analyzes the sentiment of text, returning a positive score if text is more positive than negative, a negative score
    #if text is more negative than positive, and 0 otherwise, whereby that score is computed as follows:
    #assign each word in text a value: 1 if the word is in positives, -1 if the word is in negatives, and 0 otherwise
    #consider the sum of those values to be the entire text’s score
    score = 0
    tknzr = TweetTokenizer()
    text = text.lower()
    print(self.negative)
    for each_word in tknzr.tokenize(text):
        if each_word in self.positive:
            score += 1

    for each_word in self.negative:
        if each_word in tknzr.tokenize(text):
            score += -1

    if score > 0:
        return 1
    elif score < 0:
        return -1
    else:
        return 0

if name == "main": word = str(sys.argv[1:]) bleh = Analyzer("positive-words.txt", "negative-words.txt").analyze(word) print(bleh)