Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
INF6060 Information Retrieval
Review of the University of Sheffield website search system
The aim of this coursework is for you to evaluate how a search system supports users with
finding information through searching, i.e. for the kind of information seeking where the
starting point is formulating a query. This coursework requires you to carry out a form of
‘expert’ assessment of the University of Sheffield website search system (note: make sure
you assess the website search system and not StarPlus, the University library system). You
will consider how well the search system meets the needs of its users undertaking their
typical search tasks. You are required to submit two assignments. For assignment 1, you will
create tasks and queries that you will use to test the search system for assignment 2. For
this second assignment you will produce a portfolio of 4 sets of activities and
recommendations for how the search system could be improved. As a piece of academic
work, you are expected to engage with the research literature and provide references to
justify and support your writing.
The tasks and queries (assignment 1) are worth 10% of
the overall module mark and the portfolio with recommendations (assignment 2) are worth
90% overall. Submission deadline for the tasks and queries is 10am Monday 31st October
2022 viaTurnitin. Submission deadline for the portfolio with recommendations is 10am
Monday 23rd January 2023 via Turnitin.
Assignment 1: Tasks and queries (10%)
In weeks 2 and 3 you will learn how to create tasks and queries for testing search systems.
For this assignment, you are required to create tasks and queries that you can use to test the
University of Sheffield website search system. There is no word count but you should aim to
be succinct.
Tasks and
queries
What we are looking for Mark
Simulated work
tasks (SWT)
Two SWT that are realistic for the search system (i.e. based
on what typical users might do). The situation should be
different for each SWT.
35
Search tasks
One search task for each of the two SWT tasks. The search
task should be categorised according to the research literature
(e.g. Toms, Kellar etc). The two search tasks should be
different types (e.g. if one is a specific item the other should be
general topical).
35
Queries
10 queries for each search task. These queries should be
categorised and sufficiently different that you could test
different aspects of the search performance.
20
References All use of the research literature should be acknowledged and
cited using the APA style.
5
Presentation
Good use of English and clear presentation style.
5
Assignment 2: Portfolio of activities with recommendations (90%)
For this assignment you will develop a tokenisation and text processing strategy and evaluate
the University of Sheffield website search system for how well the search system meets the
needs of its users undertaking their typical search tasks. In weeks 4-8 you will learn how to
conduct different types of evaluation (heuristic evaluation, measuring retrieval system
effectiveness, and supporting the users’ search process) and how to develop text tokenisation
and processing strategies. Throughout the module, but particularly in weeks 9-12, you will
learn about what is state-of-the-art in information retrieval that you can use to help you develop
recommendations.
You are required to produce a portfolio that includes
1. The tasks and queries you used to test the search system
2. The results of your four activities
3. Recommendations based on your activities and what is considered to be best practice
in the information retrieval research literature.
Each of these sections is described in more detail next.
(1) Tasks and queries. You must state all the tasks and queries that you have used for the
four activities. You can reuse the tasks and queries created for assignment 1. However, as
you become more familiar with the module content and the website to be evaluated you may
wish to revise and extend the assignment 1 tasks and queries. It is also recommended that
you revise your original tasks and queries in response to the feedback received for assignment
1. Additionally, you should also briefly explain why you have made changes to the tasks and
queries.
There is no word limit for the tasks and queries but you should keep your entries succinct. We
have suggested an approximate number of words as a guideline.
Portfolio - section 1 What we are looking for Marks
Tasks and queries All tasks (simulated work tasks and search
tasks) and queries used in the activities.
(guideline up to 100 words per simulated
work task)
1 mark
deducted from
presentation if
not included.
Reflection
A brief explanation of any changes made to
the assessment 1 tasks and queries
(guideline 100 words)
1 mark
deducted from
presentation if
not included.
(2) Activities 1 - 4: You MUST conduct ALL 4 activities. The minimum requirements for
each activity is stated in the table below. You should aim to go beyond this. Some ideas to
help you develop and extend your activities include (i) use more tasks and queries, (ii)
compare against website search systems from other universities, (iii) test on different
devices etc.
There is no word limit for the activities but you should keep your entries succinct. We suggest
500 words per activity as a guideline.
Portfolio - section 2 What we are looking for (minimum
requirements)
Marks
Activity 1: heuristic
evaluation
A table that lists the 10 Nielsen heuristics and
provides a brief description of whether the
search interface deals with the heuristic
effectively for 1 of your SWT.
A brief summary of the findings.
15
Activity 2: text tokenisation
and processing strategy
A logical strategy for processing and tokenising
text documents to be indexed by the search
system. You should justify any assumptions you
make when generating this strategy and give
concrete examples of how your text processing
strategy would transform original raw text
documents.