Problem

The goal of assignment 4 is to replicate the material we covered in the Text Analysis Lab. You will be processing and analyzing a corpus of text by using different kind of techniques we learned in class. The data you will be working with contains information on social service agencies. You can find the data in the wager_ada_2018/shared folder. It is called socialservices.csv.

1. Load the dataset into a dataframe

2. How many facilities and types of facilities are in this dataset? List unique facility types and names

3. Clean and process the column textfromurl by executing following steps

• Form a corpus of text

• Clean and normalize text

4. Create a bag of words and set of features, including

• Removal of stop-words
• Stemming and lemmatization
• Adding context by N-grams
• Add frequency weights for word counts

5. Count of the number of times that each of the words appears in the corpus

6. Generate 15 topics with the top 20 words.