Scenario
You have just started working as a data miner/analyst in the Analytics Unit of a company. The Head of the Analytics Unit has brought you a data set [a welcome present ;-))]. The data set includes two files: description of the attributes and a table with the actual values of these attributes. The Head of the Analytics Unit has mentioned to you that this is some sort of demographic data that a potential client has provided for analysis. The Head of the Analytics Unit would like to have a report with some insights about that data, that she could deliver to the client. Your tasks include:
- understanding the specifics of the data set
- extracting information about each of the attributes, possible associations
between them and other specifics of the data set.
The tasks in the assignment are specified below.
Data sets
The description of the attributes is the same for all students and comes in a tiny documentation file (download it from UTS Online). Each student is assigned an
31250 / 32130 Assignment 2 1
individual table with the actual values of these attributes. Please, download the file that is linked to your name from UTS Online.
Tasks
1A. Initial data exploration
1. Identify the type of each attribute (nominal, ordinal, interval or ratio). If it's not clear you may need to justify why you choose the type.
2. Identify the values of the summarising properties for each attribute including frequency, location and spread (e.g. value ranges of the attributes, frequency of values, distributions, medians, means, variances, percentiles, etc. - the statistics that have been covered in the lectures and materials given). Note that not all of these summary statistics will make sense for all the attribute types, so use your judgement! Where necessary, use proper visualisations for the corresponding statistics.
3. Using Rattle, GGobi or other tools, explore your data set and identify any outliers, clusters of similar instances, "interesting" attributes and specific values of those attributes. Note that you may need to 'temporarily' recode attributes to numeric for GGobi. In the report include the corresponding snapshots from the tools and explanation of what has been identified there.
Present your findings in the assignment report.
1B. Data preprocessing
Perform each of the following data preparation tasks (each task applies to the original data):
1. Use the following binning techniques to smooth the values of the Age attribute:
- equi-width binning
- equi-depth binning.
In the assignment report for each of these techniques you need to illustrate your steps. In your Excel workbook file place the results in separate columns in the corresponding spreadsheet. Use your judgement in choosing the appropriate number of bins - and justify this in the report.
2. Use the following techniques to normalise the attribute Age:
- min-max normalization to transform the values onto the range
[0.0-1.0].
- z-score normalization to transform the values.
In the assignment report provide explanation about each of the applied techniques. In your Excel workbook file place the results in separate columns in the corresponding spreadsheet.
31250 / 32130 Assignment 2 2
c. Discretise the Age attribute into the following categories: Teenager = 1-20; Young = 21-30; Mid_Age = 31-45; Mature = 46-65; Old = 66+. Provide the frequency of each category in your data set.
In the assignment report provide explanation about each of the applied techniques. In your Excel workbook file place the results in a separate column in the corresponding spreadsheet.
d. Convert Education Level variable into binary variables [with values "0" or "1"].
In the assignment report provide explanation about the applied binarisation technique. In your Excel workbook file place the results in separate columns in the corresponding spreadsheet.