Ask Question, Ask an Expert


Ask Programming Language Expert

A. Introduction

Sentiment analysis is a subfield of NLP concerned with the determination of opinion and subjectivity in a text, which has application in analysis of online product reconsiders, recommendations, blogs, and other kinds of opinionated documents.

In this assignment you will be developing classifiers for sentiment analysis of movie reviews by using Support Vector Machines (SVMs), in the way of the paper by Pang, Lee, and Vaithyanathan [1], which was the foremost research on this topic. The goal is to develop a classifier that performs sentiment analysis, assigning a movie review a label of "positive" or "negative" that predicts whether the author of the review liked the movie or disliked it.

You might use Java or Python programming and scripting languages of your choice for this assignment, but for the machine learning you should use SVMlight (section D).

B. Data

The data (accessible on the course web page) consists of 1,000 positive and 1,000 negative reviews. These have been divided in training, validation, and test sets of 800, 100, and 100 reviews, respectively. In order to promote you not to optimize against the testing set while building your classifiers, the testing data will not be immediately available.
The reviews were obtained from Pang's website [2], and then part-of-speech tagged by using a bidirectional Maximum Entropy Markov Model [3, 4].

Each document is formatted as one sentence per line. Each token is of the format word/POStag, where a "word" also includes punctuation. Each word is in lowercase. There is sometimes more than one slash in a token, such as in preparer/director/NN.

C. Baseline system

For a baseline system, think of 20 words that you think would be indicative of a positive movie review, and 20 words that you think would be indicative of a negative reconsider.

To develop the baseline classifier, take this approach: given a movie review, count how many times it contains either a positive word or a negative word (token occurrences). Allocate the label POSITIVE if the review holds more positive words than negative words. Assign the label NEGATIVE if it contains more negative words than positive words. If there are an equal number of positive and negative words, it is a TIE.

D. Machine learning

The machine learning software to be used is SVMlight [5], which learns Support Vector Machines for binary classification. It is available for UNIX systems, Windows, and Mac OS X.

You will require reading the documentation on the SVMlight website in order to figure out how to use the software. To test whether you know how to use it, it might be helpful to first create a small, "toy" dataset by hand, and then train and test the SVM on it. When training the classifier, choose the option for classification:

-z {c,r,p} - select between classification (c), regression (r), and
preference ranking (p)

A training file is of the format:

.=. : : ... : #
.=. +1 | -1 | 0 |
.=. | "qid"

Since we are doing binary classification, the value of should be +1 or -1.

Every feature (which might be expressed as an integer or a string) is associated with a value, which is a floating-point number. If you want a feature to be binary-valued, you may use values of 0.0 and 1.0.

With binary features, it is not necessary to comprise an explicit representation feature of features that do not occur. For illustration, suppose a document contains 100 different words out of a vocabulary of 50,000 possible words. If you are using binary features, it suffices to include a feature with a value of 1.0 for each of the words that do occur. You do not have to include a feature with a value of 0.0 for each of the 49,900 words that do not appear in the document.

You do not need to perform smoothing.

E. Feature sets

Use these feature sets for training and testing your classifier:

1. unigrams
2. bigrams
3. unigrams + POS
4. adjectives
5. top unigrams
6. optimized

Detailed explanation:

1. unigrams: use the word unigrams that occurred >= 4 times in the training data. Let this quantity be N.
2. bigrams: use the N most-frequent bigrams.
3. unigrams + POS: use all combinations of word/tag for each of the unigrams in (1). Since a word may occur with multiple tags, the quantity of this type of feature will be greater than N.
4. adjectives: use the adjectives that occurred >= 4 times. Let this quantity be M.
5. top unigrams: use the M most-frequent unigrams.
6. optimized: select any combination of features you would like, to try to produce the best classifier possible. For ex, you might choose different cut off values for frequencies of dissimilar types of features. You could also make entirely new types of features. You could also try dissimilar settings for training the SVM. The optimized classifier should be produced through a process of repeatedly training the classifier and computing its performance on the validation set.

F. Evaluation

Train the SVMs on the training data and perform preliminary tests on the validation data. To appraise your classifiers, compute the accuracy rate on the testing data, which is percentage of movie reviews correctly classified. For the baseline classifier, also find out the number of ties.
Appraise your classifiers on testing data when it is released. Don’t further optimize your system based on performance on the testing data.

G. Turn in

Produce a document that states:

- Short descriptions of the attached files
- A list of the positive and negative words selected for your baseline system
- Performance of the baseline system on the test set
- A table listing the number of divergent features for each feature set. Since the split of the data into training and testing is not exactly the same as Panget al.¡¦s, the quantity of different features will be similar, but not identical.
- A table of performance of classifiers on validation set and test set
- A written comparison of your results with Pang et al.'s (minimum 5 lines)
- Construct a table listing the 50 most-frequently misclassified reviews (across all 6 classifiers) in the validation set, and the number of classifiers by which they were misclassified. For instance, the review cv808_12635.txt may have been misclassified by 4 classifiers. Illustrate 5 different attributes of the frequently misclassified reviews, showing excerpts from 2 reviews for each attribute. For each of these attributes, describe a probable feature that could be added to improve performance.

H. Submission:

A compressed directory, containing:

- All source code
- One ex of a feature file that you produced
- Your written document
- Any additional files that you would like to attach

Programming Language, Programming

  • Category:- Programming Language
  • Reference No.:- M91286

Have any Question? 

Related Questions in Programming Language

Concurrent programming task ydetailsmany banking systems

CONCURRENT PROGRAMMING TASK YDETAILS Many banking systems allow multiple cardholders access a single account (e.g. a business account). The consequence of this is that is possible for more than one person to attempt to w ...

1 construct a markov algorithm that will reverse the order

1) Construct a Markov Algorithm that will reverse the order of an input string that consists of zero or more upper case letters. ABCDE should become EDCBA, AB should become BA, A should stay A, and A should stay A. NOTE: ...

Purposethe purpose of this assignment is to practice using

Purpose: The purpose of this assignment is to practice using loops, selections, and other basic features of Java in order to solve a real-world problem. Program Description: The Jeddah Broker Company is one of the compan ...

Shell programmingyou have created the directory structure

Shell Programming You have created the directory structure and some base files to be used by the Web server and Web site. For the Web site to be created in a production environment, you need to package your commands in a ...

The given machine description will provide the basis for

The given machine description will provide the basis for this assignment. You will create a virtual machine/operating system for the machine described below that will accept programs in the target machine language. The d ...

This assignment introduces you to the perl language and cgi

This assignment introduces you to the Perl language and CGI programming. You will use your scripting and html skills to build a Perl program that will gather user input to then generate a personal webpage. This program w ...

The briefyour small development team of ideally three

The Brief Your small development team of (ideally) three people has been asked to implement and test the Human Resource Information System desktop application. Your software product will be a database- backed desktop app ...

Integrative programmingassignment- managing a user account

Integrative Programming Assignment- Managing a User Account List Objective: To manage a user list that can be modified and saved to a text file. Inputs: • Input text file consisting of pairs of usernames and passwords, s ...

Assignmentyou will write a program that acts like a simple

Assignment You will write a program that acts like a simple calculator for binary numbers. You should read in a string of input that has the format: number operator number. Each of the numbers should be presented in two' ...


Computer Science Program- Write all of the following: main program: Call a function to open an input file. Call a function to read 3 integers in from the input file. Call a function that will find 3 normalized doubles, g ...

  • 4,153,160 Questions Asked
  • 13,132 Experts
  • 2,558,936 Questions Answered

Ask Experts for help!!

Looking for Assignment Help?

Start excelling in your Courses, Get help with Assignment

Write us your full requirement for evaluation and you will receive response within 20 minutes turnaround time.

Ask Now Help with Problems, Get a Best Answer

A cola-dispensing machine is set to dispense 9 ounces of

A cola-dispensing machine is set to dispense 9 ounces of cola per cup, with a standard deviation of 1.0 ounce. The manuf

What is marketingbullwhat is marketing think back to your

What is Marketing? • "What is marketing"? Think back to your impressions before you started this class versus how you

Question -your client david smith runs a small it

QUESTION - Your client, David Smith runs a small IT consulting business specialising in computer software and techno

Inspection of a random sample of 22 aircraft showed that 15

Inspection of a random sample of 22 aircraft showed that 15 needed repairs to fix a wiring problem that might compromise

Effective hrmquestionhow can an effective hrm system help

Effective HRM Question How can an effective HRM system help facilitate the achievement of an organization's strate