Ask Question, Ask an Expert

+1-415-315-9853

info@mywordsolution.com

Ask Programming Language Expert

A. Introduction

Sentiment analysis is a subfield of NLP concerned with the determination of opinion and subjectivity in a text, which has application in analysis of online product reconsiders, recommendations, blogs, and other kinds of opinionated documents.

In this assignment you will be developing classifiers for sentiment analysis of movie reviews by using Support Vector Machines (SVMs), in the way of the paper by Pang, Lee, and Vaithyanathan [1], which was the foremost research on this topic. The goal is to develop a classifier that performs sentiment analysis, assigning a movie review a label of "positive" or "negative" that predicts whether the author of the review liked the movie or disliked it.

You might use Java or Python programming and scripting languages of your choice for this assignment, but for the machine learning you should use SVMlight (section D).  http://svmlight.joachims.org/

B. Data

The data (accessible on the course web page) consists of 1,000 positive and 1,000 negative reviews. These have been divided in training, validation, and test sets of 800, 100, and 100 reviews, respectively. In order to promote you not to optimize against the testing set while building your classifiers, the testing data will not be immediately available.
The reviews were obtained from Pang's website [2], and then part-of-speech tagged by using a bidirectional Maximum Entropy Markov Model [3, 4].

Each document is formatted as one sentence per line. Each token is of the format word/POStag, where a "word" also includes punctuation. Each word is in lowercase. There is sometimes more than one slash in a token, such as in preparer/director/NN.

C. Baseline system

For a baseline system, think of 20 words that you think would be indicative of a positive movie review, and 20 words that you think would be indicative of a negative reconsider.

To develop the baseline classifier, take this approach: given a movie review, count how many times it contains either a positive word or a negative word (token occurrences). Allocate the label POSITIVE if the review holds more positive words than negative words. Assign the label NEGATIVE if it contains more negative words than positive words. If there are an equal number of positive and negative words, it is a TIE.

D. Machine learning

The machine learning software to be used is SVMlight [5], which learns Support Vector Machines for binary classification. It is available for UNIX systems, Windows, and Mac OS X.

You will require reading the documentation on the SVMlight website in order to figure out how to use the software. To test whether you know how to use it, it might be helpful to first create a small, "toy" dataset by hand, and then train and test the SVM on it. When training the classifier, choose the option for classification:

-z {c,r,p} - select between classification (c), regression (r), and
preference ranking (p)

A training file is of the format:

.=. : : ... : #
.=. +1 | -1 | 0 |
.=. | "qid"
.=.
.=.

Since we are doing binary classification, the value of should be +1 or -1.

Every feature (which might be expressed as an integer or a string) is associated with a value, which is a floating-point number. If you want a feature to be binary-valued, you may use values of 0.0 and 1.0.

With binary features, it is not necessary to comprise an explicit representation feature of features that do not occur. For illustration, suppose a document contains 100 different words out of a vocabulary of 50,000 possible words. If you are using binary features, it suffices to include a feature with a value of 1.0 for each of the words that do occur. You do not have to include a feature with a value of 0.0 for each of the 49,900 words that do not appear in the document.

You do not need to perform smoothing.

E. Feature sets

Use these feature sets for training and testing your classifier:

1. unigrams
2. bigrams
3. unigrams + POS
4. adjectives
5. top unigrams
6. optimized

Detailed explanation:

1. unigrams: use the word unigrams that occurred >= 4 times in the training data. Let this quantity be N.
2. bigrams: use the N most-frequent bigrams.
3. unigrams + POS: use all combinations of word/tag for each of the unigrams in (1). Since a word may occur with multiple tags, the quantity of this type of feature will be greater than N.
4. adjectives: use the adjectives that occurred >= 4 times. Let this quantity be M.
5. top unigrams: use the M most-frequent unigrams.
6. optimized: select any combination of features you would like, to try to produce the best classifier possible. For ex, you might choose different cut off values for frequencies of dissimilar types of features. You could also make entirely new types of features. You could also try dissimilar settings for training the SVM. The optimized classifier should be produced through a process of repeatedly training the classifier and computing its performance on the validation set.

F. Evaluation

Train the SVMs on the training data and perform preliminary tests on the validation data. To appraise your classifiers, compute the accuracy rate on the testing data, which is percentage of movie reviews correctly classified. For the baseline classifier, also find out the number of ties.
Appraise your classifiers on testing data when it is released. Don’t further optimize your system based on performance on the testing data.

G. Turn in

Produce a document that states:

- Short descriptions of the attached files
- A list of the positive and negative words selected for your baseline system
- Performance of the baseline system on the test set
- A table listing the number of divergent features for each feature set. Since the split of the data into training and testing is not exactly the same as Panget al.¡¦s, the quantity of different features will be similar, but not identical.
- A table of performance of classifiers on validation set and test set
- A written comparison of your results with Pang et al.'s (minimum 5 lines)
- Construct a table listing the 50 most-frequently misclassified reviews (across all 6 classifiers) in the validation set, and the number of classifiers by which they were misclassified. For instance, the review cv808_12635.txt may have been misclassified by 4 classifiers. Illustrate 5 different attributes of the frequently misclassified reviews, showing excerpts from 2 reviews for each attribute. For each of these attributes, describe a probable feature that could be added to improve performance.

H. Submission:

A compressed directory, containing:

- All source code
- One ex of a feature file that you produced
- Your written document
- Any additional files that you would like to attach

Programming Language, Programming

  • Category:- Programming Language
  • Reference No.:- M91286

Have any Question? 


Related Questions in Programming Language

Assignment- object oriented programmingthis program will

Assignment- Object Oriented Programming This program will use 2 different data structures to perform the same tasks; a vector and a dynamically sized array Write a complete C++ program that will: Declare a vector of inte ...

Write a program which1 asks the user to enter a positive

Write a program which: 1. Asks the user to enter a positive integer greater than or equal to 0 2. Validates that the entry is a positive integer 3. Outputs the digits in reverse order with a space separating the digits 4 ...

Programming taskswhile working on the tasks below you are

Programming tasks While working on the tasks below, you are free to request assistance on D2L at: Communications / Discussions / Assignments (or other appropriate sub-topic) These tasks entail modifying your prior code f ...

Assignment1 you are a manager who is employed by a game

Assignment 1. You are a manager who is employed by a game production company. Your team is responsible for coding one of the game modules. You have two newly hired programmers working for you who believe that the followi ...

In this programming assignment you are asked to simulate

In this programming assignment, you are asked to simulate the recursive factorial function given in the class. Your program is to be a nonrecursive version of the factorial function implemented using a stack. Your progra ...

Assignmentthe csit racing club is a group that runs amateur

Assignment The CSIT Racing Club is a group that runs amateur car racing events throughout the US. In This project, write a program that will help the club determine the winner of their Fall Rally Race. You will need to d ...

Create a base class that will include the properties of

Create a base class that will include the properties of some type of data related to the company you work for or a favorite hobby. For example, I work for a healthcare provider, so my base class might be "Medical Facilit ...

Question 1what is overriding please use code to explain it

Question 1: What is overriding? Please use code to explain it. (Cannot use the same code in class) Question 2: What are the benefits when inheritance is used if multiple classes share the same fields and methods? Questio ...

Assignmentfor questions 2-9 you must submit your code

Assignment For questions 2-9 you must submit your code together with the output of running your code on some sensible sample input. Submit the input and the output of your program along with your code. Question 1 Conside ...

1 design a class called complex the class enables

1) Design a class called Complex. The class enables operations on so called complex numbers. These are numbers of the form realPart + imaginary Part *i, where the i has the value 2) Design the class to have two private m ...

  • 4,153,160 Questions Asked
  • 13,132 Experts
  • 2,558,936 Questions Answered

Ask Experts for help!!

Looking for Assignment Help?

Start excelling in your Courses, Get help with Assignment

Write us your full requirement for evaluation and you will receive response within 20 minutes turnaround time.

Ask Now Help with Problems, Get a Best Answer

Section onea in an atwood machine suppose two objects of

SECTION ONE (a) In an Atwood Machine, suppose two objects of unequal mass are hung vertically over a frictionless

Part 1you work in hr for a company that operates a factory

Part 1: You work in HR for a company that operates a factory manufacturing fiberglass. There are several hundred empl

Details on advanced accounting paperthis paper is intended

DETAILS ON ADVANCED ACCOUNTING PAPER This paper is intended for students to apply the theoretical knowledge around ac

Create a provider database and related reports and queries

Create a provider database and related reports and queries to capture contact information for potential PC component pro

Describe what you learned about the impact of economic

Describe what you learned about the impact of economic, social, and demographic trends affecting the US labor environmen