Decision Discussion Annotation Schema

Basic Definitions: Decission Discussions and their Structureal Units

A decision discussion is a part of a dialogue, comprised of a set of utterances, in which a resolution is reached after considering a particular issue or topic.

We consider a set of five structural units that may be present in decision discussions:

1. Issue under Discussion
2. Resolution Proposal
3. Resolution Restatement
4. Agreement
5. Agreement Elicitation

Issue under discussion

As any discussion, decision discussions are about a particular topic or issue. Often the issue under consideration is raised explicitly (e.g. ``color-wise, what does everybody think?''), but the topic of the discussion may also be implicitly introduced. Sometimes there are long argumentations between bringing up the issue and coming to a resolution. In long and branching argumentations, the issue may be brought up again to focus the discussion shortly before a resolution is reached.

Those utterances that best contribute to defining the issue under discussion should be annotated as Issue. As a general guideline, take the most recent mention of the issue and do not include unnecessary elaborations - although if more than one utterance is required to understand the full issue, it's fine to include more than one.

Resolution Proposal / Restatement

A decision discussion is not a mere discussion on a topic or issue, but one that reaches a clear resolution or conclusion. Such resolution, which corresponds to the decision made, is the core aspect of a decision discussion -- the minimal item that any participant would write down in a record of decisions. A decision discussion may include several proposals. The resolution is the proposal that gets accepted.

We distinguish between Resolution Proposal and Resolution Restatement:

Resolution Restatement

If the decision discussion includes an utterance that is a sort of final summary of the decision made, then that utterance should be annotated as Resolution Restatement. For instance, in a discussion like ``We could use rubber buttons. Yeah/Maybe/They are uncool but cheap/... Okay, so we'll use rubber buttons'' , the final utterance should be tagged as Resolution Restatement. Note that the resolution restatement may be elliptical ( ``sorted, we'll do that'' ).

Resolution Proposal

Those utterances where the resolution is proposed should be annotated as Resolution Proposal. For instance, in an example like the one above, the first utterance ( `` we could use rubber buttons'' ) should be annotated as Resolution Proposal.

When there is no summarizing resolution restatement but just a proposal that gets accepted, the right class to use is Resolution Proposal (instead of Resolution Restatement).

[ is this really how we want to do it?? ] Note that sometimes a question may be the best candidate utterance to define the resolution proposal. For instance, in a dialogue like ``A: Shall we use a plastic case? B: Yeah'', A's question (which might also contribute to define the issue) is probably the utterance that best contributes to define the resolution proposal (namely to use a plastic case). [_isn't this also an agreement elicitation?_]

Negative decisions (``A: Shall we use a plastic case? B: No'') are trickier. Here both question and answer could be annotated as Resolution Proposal. If the negative decision leads to a positive one clearly defined by other utterances (``A: Okay, then let's use titanium. B: That sounds good''), then the negative decision can be treated as an alternative proposal and be ignored (with the positive decision being annotated as Resolution Proposal).

If you find any particularly tricky examples, make a note so that we can discuss them later.

Agreement

When a resolution is reached, this is generally followed by expressions of agreement. In fact, modulo participant role factors, a proposal counts as the resolution of a decision discussion only when it is agreed upon by the participants.

Any utterance that shows agreement either with the resolution proposal or the resolution restatement should be annotated as Agreement. Note that here we are interested in utterances whose primary function is to indicate agreement (such as ``Okay'', ``Yeah'', ``That sounds good'') and not in utterances that provide arguments in support of the resolution (even though they may indicate agreement as well).

Ageement elicitation

Sometimes, usually towards to end of a decision discussion, we can find a sort of check question calling for final agreement (such as ``should we go with that?'' or ``everybody happy with this?''). This kind of utterances should be annotated as ElicitAgreement.

Annotating Decision Discussions

To browse and annotate a meeting you will use NOMOS. Once NOMOS is running:

1. Open the New Session wizard (Session > New)
2. Select (or enter) your name as (new) annotator
3. Select the task deepDecisionDiscussions
4. Select the corpus and the file of the meeting you want to access
5. Selelct the models daSegment and DecisionDiscussions
6. Select the perspective DecisionDiscussionsWithSubgroups

Besides a track for each of the speakers, the new session will have two additional tracks: a DecisionDiscussions track that displays the AMI annotation of decision discussion regions (which can serve a guideline), and a DecisionSubgroups track which you will use at a later stage of your annotation.

Annotation Stage 1: Identifying and Describing Decision Discussions

At this initial stage, you will be using NOMOS to view and listen to the meetings, but you are not required to enter any annotations into NOMOS until the next stage. During this first phase, it is important that you take notes as this will speed up the second phase of the annotation.

Identify the Decision Discussions in the meeting. Recall from the definitions above that a DD is a discussion whereby participants reach a conclusion after considering a particular topic or issue. Once you have identified a DD and have listed to it, you should write down the following information:

1. Make a note of the rough starting and end times of the DD (this will help you find it in stage 2)
2. Write down a succinct description of what the issue is
3. Write down a succinct description of what the resolution is (the decision made)
4. If there is a resolution restatement and/or an agreement elicitation make a note of the rough time at which they appear
5. Determine whether the DD is exemplary. If the DD seems non-standard to you for whatever reason - for instance, if you are not 100% sure whether a decision has actually been made, then you should consider that DD non-exemplary.

Beware that sometimes participants make provisional decisions which get overruled later on in the meeting - but as they are still decisions (at the time they are made), they should be considered as such.

Annotation Stage 2: Annotating the Identified Decision Discussions

At this stage, you are going to use NOMOS to annotate the utterances that contribute to defining the decision discussions that you identified in stage 1.

The first thing you need to do is to create your own DecisionDiscussions track as follows:

1. Right-click on any empty space in the DecisionDiscussions track and select Create Decision Discussion
2. Enter any character on any of the fields and click Done.
3. Save the session, close it (close the session, not NOMOS!), and use Session > Recent Sessions to reopen it again. This will have created a second track Decision Discussions yourname
4. Delete the dummy decision discussion you just created by right-clicking on it and selecting Delete

Now you are ready to start annotating. For each of the DDs that you identified in stage 1, you need to go through the following steps:

1. Create a Decision Discussion Object
   • locate the part of the dialogue where the decision discussion takes place using your notes from stage 1.
   • Right-click on the Decision Discussions yourname track and select Create Decision Discussion.
   • After a few seconds a dialogue box will appear. On the Issue and Resolution slots enter the succinct description of the issue and the resolution that you wrote down in stage 1, respectively.
   • If you determined that the DD was non-exemplary, then enter "false" in the isExemplary slot; otherwise leave that slot empty.
   • All other slots can be left empty.
2. Create the Subgroup Nodes: Next you will create the Decision Subgroups in the DecisionSubgroup track and link them to the Decision Discussion on the Decision Discussions yourname track.
   • Right-click on the Decision Subgroups track and select Create Decision Subgroup.
   • Choose the type of subgroup you want to create and when the dialogue appears enter a space in the description field (you will update this later).
   • Typically you want to create subgroup nodes at least for Issue, Resolution Proposal, and Agreement; if in stage 1 you identified a Resolution Restatement and/or an Elicit Agreement then you also need to create subgroups for these.
   • Link the Subgroup nodes to the Decision Discussion.
   • Note that each DD can have at most one subrgoup of each kind (although several utterances can be linked to one subgroup node)
3. Annotate utterances as part of the subgroups: The last step is to identify which utterance(s) contribute to define each subgroup and link them to the appropriate subgroup node.
   • Which utterance(s) bring up the issue? Select them and link them to the Issue subgroup (in red); In which utterance(s) is the resolution proposed? Select them and link them to the Resolution Proposal subgroup (in yellow); Which utterances express simple agreement with the resolution? Select them and link them to the Agreement subgroup. And so on for Resolution Restatement and Elicit Agreement....
   • When determining if an utterance should be marked, the general guideline to follow is to mark those utterances which necessarily and sufficiently contributed to your understanding of the corresponding component of the decision discussion. Do not mark utterances that are superfluous or that elaborate on the main point (for instance, arguments in favor of the resolution should not be marked as belonging to the resolution proposal itself).
4. Fill in the description slot of the Issue, Resolution Proposal and Resolution Restatement subgroups: Now that you know which utterances belong to each subgroup, you need to go back to the subgroup nodes and edit the description slot.
   • For each of these subgroups, click on the subgroup object and select Edit Object.
   • In the description slot, write the word(s) from the utterance that you annotated as belonging to that subgroup that best describe it -- write down the words exactly as they appear in the utterance please!
   • If more than 1 utterance has been annotated as belonging to one subgroup and you need to write down words from several utterances into the description slot, then use a plus sign (+) to separate the word(s) that belong to different utterances
   • Some examples (what should be written into the description slot is indicated with square brackets):
     • Issue "Um, [what is the purpose of the light?]"
Resolution Proposal "[Banana] I think"
     • Issue "How [how should we uh supply the thing with energy]"
Resolution Proposal "Maybe it's better to to include [rechargeable batteries]"
     • Issue "but but a [hard rubber] like this?"
Issue "[Or softer rubber]"
here you would write it like this "hard rubber + or softer rubber"
Resolution Proposal "Uh [hard rubber] I think"

-- RaquelFernandez - 16 Apr 2008