Project: Analyse peer to peer user reputation statistics

Project Definition

1. Background and Rationale for the project
In order to give each of our users a "reputation score" relative to others, we need to know the mean, mode, and median etc of ratings and reputation on each service utilising reputation.

2. Project goals and Objectives
Given reputation data provided in a database format from popular services (e.g. number of stars, number of reviews, age of account, location etc) for all users, we need to compute the mean, mode and median reputations and use these to "rate" each of our users.

This data analysis should be easily extended - e.g. further break down down by country and city, where possible (so we write queries like like what is the mean Ebay reputation for users in Sydney?).

3. Project Resources
This project relies on the Project Title "Programmatically gather peer to peer user reputation data (part 1 of 2)" so discussion needs to be had about how the data is stored and an appropriate method of writing queries to extract useful data.

4. Desired Outcomes/Deliverables
Mean, mode, median (and other useful data) provided for all services provided by part 1 of the project. If this data is pre-calculated, the queries used to produce the data must also be provided so we can constantly update the data.

5. Add any Other information you wish on the project

Project description

Project - Programmatically gather peer to peer user reputation data

Project Definition

1. Background and Rationale for the project

In order to give each of our users a "reputation score" relative to others, we need to know the mean, mode, and median etc of ratings and reputation on each service utilising reputation.

2. Project goals and Objectives

We need to spider popular services such as ebay for all their profiles recording the reputation data (number of stars, number of reviews, age of account, location etc) from each users account.

3. Project Resources

There are many tools that can do the scraping

2. Project goals and Objectives

We need to spider popular services such as ebay for all their profiles recording the reputation data (number of stars, number of reviews, age of account, location etc) from each users account.

3. Project Resources

There are many tools that can do the scraping and data collection so research can be done into which is most suitable.

Or the students could write their own script which is likely to be more versatile (and without subscription costs which we unfortunately have no budget for).

Main sites we need to gather reputation from (in strict order of priority):

Must have (in strict order of priority):
Stackoverflow
Github
Linkedln
Airbnb
Should have:
Airtasker
Facebook
Ebay
Upwork
Freelancer
TaskRabbit
Quora
TripAdvisor
Trustpilot
Reddit

4. Desired Outcomes/Deliverables
Database of data for many (or all) services listed above (in that order of priority) with all of the reputation data captured.
MySQL database is preferred and recommended data store (although we are open to suggestions of better alternatives - Elastic Search?).

- Number of reviews
- % positive feedback
- Join date
- Location
- Number of positive reviews
- Number of neutral reviews
- Number of negative reviews
- Number of followers
- Number of collections
- Number of guides
- Number of reviews
- Number of views

5. Add any Other information you wish on the project
Some services may provide their own APIs that can be used to gather their data in an easier and more efficient war.