Annotator Training Overview

May 6, 2013, Nigel Ward


Thank them for helping us with our research.

Review the compensation ($10 per hour), and projected time required
(20 hours).

Briefly review our vision of search in social multimedia.

Explain why we need their help (to tag similar sets, so we can tune
our algorithms).


Step 1: Familiarization with the Corpus and Elan

Show them 20 seconds of sample video (dialog 000).  Explain the corpus.

Show them how Elan works, show them how to annotate a couple regions.

Let them try it out for themselves.

Give them the How To Tag Using Elan document, and have them try out
all the commands.



Step 2: Familiarization with the Task

Give them the Annotators Guide and ask them to read the Guidelines
section.  Give them lots of time, then answer any questions.

Have them label the first 5 minutes of one of the nicest dialogs (for
labler 2 use 004, labeler 3 006, 4 008, and 5 012).

Then review the tags they came up with, and discuss them with them.
Call Nigel at x6827 to come discuss also, if he's around. Unless
they're way out of line or misunderstanding something fundamental,
just praise their decisions and listen to their justifications.

Copy the .eaf file they created somewhere else, so they have a clean
start.


Then set them to work, according to the steps in the Procedure section
of the Annotators Guide.



This file is annotator-training-overview.txt in
/home/users/nigel/papers/mediaeval/instructions/ .