CMT316 Applications of Machine Learning
Applications of Machine Learning
Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
CMT316
Applications of Machine Learning: Natural Language Processing and
Computer Vision
Assessment Title: Coursework 1
Assessment Number: 1
Date Set: Monday, 6th February 2023 (Week 2)
Submission Date and Time: by Monday, 6th March 2023 (Week 6) at 9:30am
Feedback Return Date: 3rd April 2023
Extenuating Circumstances submission deadline will be 2 weeks after the submission
date above
Extenuating Circumstances feedback return will be 2 weeks after the feedback return
date above
This assignment is worth 50% of the total marks available for this module. If coursework
is submitted late (and where there are no extenuating circumstances):
1 If the assessment is submitted no later than 24 hours after the deadline, the
mark for the assessment will be capped at the minimum pass mark;
2 If the assessment is submitted more than 24 hours after the deadline, a mark
of 0 will be given for the assessment.
Extensions to the coursework submission date can only be requested using the
Extenuating Circumstances procedure. Only students with approved extenuating
circumstances may use the extenuating circumstances submission deadline. Any
coursework submitted after the initial submission deadline without *approved*
extenuating circumstances will be treated as late.
More information on the extenuating circumstances procedure can be found on the
Intranet: https://intranet.cardiff.ac.uk/students/study/exams-and-
assessment/extenuating-circumstances
By submitting this assignment you are accepting the terms of the following declaration:
I hereby declare that my submission (or my contribution to it in the case of group
submissions) is all my own work, that it has not previously been submitted for
assessment and that I have not knowingly allowed it to be copied by another student. I
understand that deceiving or attempting to deceive examiners by passing off the work of
another writer, as one’s own is plagiarism. I also understand that plagiarising another’s
work or knowingly allowing another student to plagiarise from my work is against the
University regulations and that doing so will result in loss of marks and possible
disciplinary proceedings1.
In this coursework, students demonstrate their familiarity with the topics covered in the
module via two separate parts with equal weight (first part: 40%; second part: 60%).
Part 1 (40%)
In Part 1, students are expected to answer two practical questions.
1. Your algorithm gets the following results in a classification experiment. Please
compute the precision, recall, f-measure and accuracy *manually* (without
the help of your computer/Python, please provide all steps and formulas).
Include the process to get to the final result. (15 points)
Id Prediction Gold
1 True True
2 True True
3 False True
4 True True
5 False True
6 False True
7 True True
8 True True
9 True True
10 False False
11 False False
12 False False
13 True False
14 False False
15 False True
16 False False
17 False False
18 True False
19 True False
20 False False
2. You are given a dataset (named “real_state”) with different house properties
(dataset available in Learning Central). Your goal is to train machine learning
models in the training set to predict the house price of a unit area in the test
set. The problem should be framed as both regression and classification. For
regression, the house price of a unit area is given; for classification, there would
be two labels (expensive and not-expensive) depending on the house price of a
unit area: expensive if it is higher or equal to 30, and not-expensive if it is lower
than 30.
The task is therefore to train two machine learning models (one regression and
another one classification) and check their performance. The student can
choose the models to solve this problem. Write, for each of the models, the
main Python instructions to train and predict the labels (one line each, no need
to include any data preprocessing instructions in the pdf) and the performance
in the test set in terms of Root Mean Squared Error (regression) and accuracy
(classification). While you will need to write the full code to get to the results,
only these instructions are required in the pdf. (25 points)
Part 2 (60%)
In Part 2, students are provided with a text classification dataset (named “bbc_news”).
The dataset contains news articles split into five categories: tech, business, sport, politics
and entertainment. Based on this dataset, students are asked to preprocess the data,
select features and train and evaluate a machine learning model of their choice for
classifying news articles. Students should include at least three different features to train
their model, one of them should be based on some sort of word frequency. Students can
decide the type of frequency (absolute or relative, normalized or not) and text
preprocessing for this mandatory word frequency feature. The remaining two (or more)
features can be chosen freely. Then, students are asked to perform feature selection to
reduce the dimensionality of all features.
Note: Training, development and test sets are not provided. It is up to the student to decide
the evaluation protocol and partition (e.g., cross-validation or pre-defining a training,
development and test set). This should be explained in the report.
Deliverables for this part are the Python code including all steps and a report of up to 1200
words. The Python code is a single solved and run jupyter notebook (.ipynb) with clear
execution paths and comments, i.e. make sure to restart the kernel and execute each cell
such that all outputs and figures (if applicable) are visible. The code should take the
dataset set as input, and output the results according to the chosen evaluation protocol.
The code will consist of 25% of the marks for this part (15 points) and the report the
remaining 75% (45 points). The code should contain all necessary steps described above:
to get the full marks for the code, it should work properly and clearly perform all required
steps. The report should include:
1) Description of all steps taken in the process (preprocessing, choice of features,
feature selection and training and testing of the model). This description should be
such that one could understand all steps without looking at the code (15 points -
The quality of the preprocessing, features and algorithm will not be considered
here)
2) Justification of all steps. Some justifications may be numerical, in that case a
development set can be included to perform additional experiments. (10 points - A
reasonable reasoned justification is enough to get half of the marks here. The
usage of the development set is required to get full marks)
3) Overall performance (accuracy, macro-averaged precision, macro-averaged recall
and macro-averaged F1) of the trained model in the dataset. (10 points - Indicating
the results, even if very low, is enough to get half of the marks here. A minimum of
65% accuracy is required to get full marks)
4) Critical reflection of how the deliverable could be improved in the future and on
possible biases that the deployed machine learning may have. (10 points - The
depth and correctness of insights related to your deliverable will be assessed)
The report may include tables and/or figures.
Learning Outcomes Assessed
This coursework covers the six LOs listed in the module description. Specifically:
Part 1: LO1, LO2
Part 2: LO1, LO3, LO4, LO5, LO6
Criteria for assessment
Credit will be awarded against the following criteria.
➢ Part 1. The main criteria for assessment is based on the correctness of the
answers, for which the process is also required. Full marks will be given for
answers including both the correct answer and a correct justification or
methodology.
Part 1 – Q1 (15 points)
Precision
(4 points) • 0 point – no calculation process
• 1-3 points – depending on specific errors in calculation
process
• 4 points – Correct answer and correct calculation process
Recall
(4 points)
f-measure
(4 points)
Accuracy
(3 points)
• 0 point – no calculation process
• 1-2 points – depending on specific errors in calculation
process
• 3 points – Correct answer and correct calculation process
Part 1 – Q2 (25 points)
Classification
(12 points)
• 0 point – no Python instructions to train and predict the
labels
• 1-5 points – major errors in the Python instructions
• 6-11 points – minor errors in the Python instructions
• 12 points – good accuracy and correct Python instructions
Regression
(13 points)
• 0 point – no Python instructions to train and predict the
labels
• 1-5 points – major errors in the Python instructions
• 6-12 points – minor errors in the Python instructions
• 13 points – good Root Mean Squared Error (RMSE) and
correct Python instructions
➢ Part 2. This part is divided into Python code (25%) and an essay (75%). The code
will be evaluated based on whether it works or not, and whether it minimally
contains the necessary steps required for the completion of Part 2. Four items will
be evaluated in the essay, whose weights and descriptions are indicated in the
assessment instructions. The main criteria to evaluate those items will be the
adequacy of the answer with respect to what was asked, and the justification
provided.
Part 2 (60 points)
Python code
(15 points)
• 0 point – no/little code and comments
• 1-7 points – major errors/missing steps in the
code and comments
• 8-14 points – minor errors/missing steps in
the code and comments
• 15 points – the code runs and contain all
necessary steps and comments
Essay
(45 points)
Description
(15 points)
• 0 point – no description
• 1-7 points – mostly unclear descriptions of the
steps
• 8-14 points – mostly clear descriptions of the
steps but with some unclear ones
• 15 points – clear descriptions of the steps
Justification
(10 points)
• 0 point – no justification and development set
• 1-5 points – major issues in the justification
and/or the development set
• 6-9 points – minor issues in the justification
and/or the development set
• 10 points – Reasoned justification and good
use of the development set
Performance
(10 points)
• 0-5 points – no or missing performance
measures
• 6-9 points – reasonable performance
evaluation with accuracy <= 65%
• 10 points – good performance evaluation with
accuracy > 65%
Reflection
(10 points)
• 0 point – no reflection and discussion on
biases
• 1-5 points – major issues in the reflection
and/or discussion on biases
• 6-9 points – minor issues in the reflection
and/or discussion on biases
• 10 points – in-depth and correct reflection and
discussion on biases