Action Item Annotation Schema

We first present a basic definition of what an action item is. Then we present a two step process for annotating action items in meeting conversations.

Basic Definitions

An _action item_ is an concrete future action which is to be performed by one or more people and which emerges in the discussion of a project planning meeting. The simplest way to determine whether or not an action item has emerged in a discussion is to ask yourself whether or not any of the meeting participants might write down the action in their to-do list or diary. For example, if Marcia says, “So, Greg, can you have those figures for me by next week?” and Greg replies, “That should be doable,” you can imagine that Greg might soon make a note to himself that says, “Have figures by next week.” This is a clear instance of an action item.

An _action item discussion_ is a part of a meeting, comprised of a set of utterances, in which the discussion of an action item occurs. For every action item in a meeting, there must be at least one (possibly more, see below) action item discussion.

In Stage 1, you will identify and annotate the action items. In Stage 2, you will identify and annotate the action item discussions.

Annotation Stage 1: What are the Action Items?

The following is a summary of your objectives for Stage 1 of annotating a meeting for action items.

1. Identify the set of action items that emerged in the meeting
2. For each action item...
   1. Determine if the action item is exemplary
   2. Create a textual description of the action
   3. Create a textual description of the responsible parties, and identify their IDs (if this is possible)
   4. Create a textual description of the timeframe (if there is one)
3. Make a note of any times during the meeting when an action item has been discussed.

Note that you will be using NOMOS to view and listen the meetingsin this stage, but you are not required to enter any annotations into NOMOS until the next stage.

Here is an example of what you should have created by the end of Stage 1. In this example, the people are listed along with their corpus ID tags. The action items for each person are listed next to each person along with a timeframe (if there is one). Action items for all the participant have also been listed, as well as action items for which ownership is indeteminable. If an action item is exemplary, it has been underlined. Additionally, a list of questionable action items has been listed for future reference. Last, notes have been taken as to the location in the meeting where discussion of the action items has occurred. This last step is important as it can be used in Stage 2 to speed up the annotation process.

We discuss the specifics of the Stage 1 process below.

Identifying Action Items

Use the following criteria to determine if an action item has emerged in the discussion:

Required attributes:

• The content or topic of discussion is a concrete ("actionable") future action, task, or event requiring someone's participation
• The discussed action is significant enough to warrant its writing down in any participant's to-do list or diary

Exemplary attributes:

• There is an individual or set of individuals who are present in the meeting and clearly accept responsibility to carry out the action (or to ensure that it is carried out by someone else not present in the meeting)

Optional attributes:

• There is a time, date, or other temporal specification of when the action is expected to be performed
• There is an exchange of dialogue which confirms the commitment of the responsible parties

Exemplary action items: Evidence of all the required attributes would constitute a action item. If the exemplary attributes are satisfied, mark the action item as exemplary. If the optional attributes are also apparent, this should make you more confident that it is an action item (in comparison to other related types of discussion) and may even suggest that the action item is exemplary.

Implied information and intentions: Sometimes information is implied and not explicitly evident in the words that are said. If any of the above attributes are implied, it is still appropriate to consider this as evidence. Concentrate on understanding the meanings and intentions being conveyed by the participants, not necessarily on their explicit wording.

Multi-faceted actions: Sometimes actions are complex and involve multiple components or sub-actions. In this case, you should try to identify each atomic action and mark each as an individual action item. The best way to identify an atomic action is to consider whether you would write the action on one line or two separate lines in your to-do list. Each line you would write should represent one individual action item. If you are still unsure about whether it is one or two actions, err on the side of making two instead of one.

Repeated action items: Sometimes action items are brought up multiple times in a meeting. In this case, you should only mark one action item. You will identify the different dialogues where it emerges in Stage 2.

Ongoing tasks and commitments: Sometimes a task which emerges in discussion is in fact an ongoing one which was committed to previously. In this case, you should determine the conversational intentions of those who are discussing it. If one of their conversational goals is to confirm or re-establish commitment to the action, this is an action item. However, if they are simply informing others of the status of the task, this is not an action item. Your focus should be on identifying the concept of commitment and responsibility in the intentions of the participants.

Different individual perspectives on the same action item: Coming soon...

Describing the Action Item

Once you have identified an action item and determined if it is exemplary, you should now write down a set of textual descriptions for the action item.

Action description: First, you must create a description of the task that is to be performed. You should keep your description somewhat succinct, reflecting how one would likely write it down in their meeting notes. It is common (though not necessary) to include an action word with the necessary arguments, plus perhaps a qualifier. For example: "send the write-up to Mark" or "make three XML examples". Other times, a noun phrase will suffice, for example, "quarterly report".

Responsible parties: Then, create a description of the responsible parties for the action item if you can determine specifically who they are. If everyone is responsible, write "everyone". Otherwise, if you can determine the ID(s) of the individual(s), mark these down, together with the individual word or phrase which corresponds to that individual (sometimes a name "Mark", sometimes just "you" or "I"). If you cannot determine the ID, just mark down that word or phrase (in this case, probably the name of someone outside the meeting). If you cannot determine the owner (though you should be sure that there is one), use the phrase "unknown".

Time frame or due date: Last, create a description of the time frame for the action item, if there is one. This would likely appear as a modifier to your description of the action. For example: "by Friday", "between now and next time", or "soon". You should always create descriptions which match what you think someone might write in their to-do list or diary.

Annotation Stage 2: Which utterances contribute to defining the Action Item?

In Stage 2, your goal is to locate the times in the meeting when action items were discussed and to select the utterances in the dialogue which directly contribute to your understanding of the action items along four dimensions: description, ownership, timeframe, and agreement. You will also extract relevant subphrases from these utterances. In this stage, you will use NOMOS to make your annotations. To summarize, you will perform the following steps:

1. Identify each individual emergence of an action item in discussion, then for each of these...
   1. Group the description utterances
   2. Write the extracted description sub-phrases
   3. Group the ownership utterances
   4. Write the extracted ownership sub-phrases
   5. Group the timeframe utterances
   6. Write the extracted timeframe sub-phrases
   7. Group the agreement utterances
   8. Mark the agreement and commitment properties of each agreement utterance

We now present the details on how to perform these steps.

Identifying individual Action Item discussions

When performing the Stage 2 annotations, you should proceed chronologically through the meeting one again, this time locating any places in the meeting when action items are discussed. You will then make annotations of each action item discussion that you encounter before proceeding to the next one. Note that an individual action item might emerge in the discussion multiple times. In this case, you should consider each emergence as a separate group of utterances that should be annotated along the four dimensions (see notes below).

One or two discussions?: Sometimes the description of an action item may be spread over several utterances which may also be spoken by different people. These utterances may sometimes even be separated by short sub-dialogues that are not part of the action item discussion. This is fine, as long as there are not two complete separate action item discussions involved. If it is unclear whether there is one or two separate discussions, consider the attentions of the participants. If their attention drifted away from the specific action item such that bringing it back to their attention required an explicit re-introduction of the topic, then it should be considered a separate discussion.

Creating and completing the CsliActionItemGroup object

Once you have found an action item discussion, you will need to create a CsliActionItemGroup object and populate its fields, using the information you gathered for the action item in stage one, as follows.

• Place either "true" or "false" in the CsliActionItemGroup::CsliAIExemplary slot
• Place the owner description in the CsliActionItemGroup::CsliAIOwnerDescription slot
• Pull down the person objects in the CsliActionItemGroup::CsliAIOwner slot (if you know them)
• Place the description of the time frame in the CsliActionItemGroup::CsliAITimeframe slot (if there is one)
• Place the description of the action item in the CsliActionItemGroup::NOMOS-description slot

Creating and completing the CsliActionItemSubgroup objects

Next, you will create the CsliActionItemSubgroup objects and link them to the CsliActionItemGroup.

Description, Timeframe, and Ownership

Use the following guidelines for marking the description, timeframe, or ownership dimension of an action item discussion:

Selecting the utterances

Use the following guidelines for determining which utterances to select:

Necessity and Sufficiency: When determining if an utterance should be marked, you should analyze the surrounding discussion and mark those utterances which contained words or phrases which necessarily and sufficiently contributed to your understanding of the corresponding component of the action item (i.e. description, timeframe, or owner). To determine if you have marked the necessary and sufficient utterances, imagine you are a colleague of the participants but did not attend the meeting. Read only those utterances which were marked. If those utterances are enough communicate what it is that the person is supposed to do, then you have met the sufficiency condition, and you don't need to add any more utterances. Otherwise, continue to add utterances until you meet the sufficiency condition. Now, starting with the earliest utterance you have marked and proceeding forward, if that utterance could be removed and not affect the sufficiency condition, it should be removed. Once you have done this, you have met the necessity condition.

Considerations for description tagging: While staying true to the above necessity and sufficiency conditions, note that utterances which communicate non-critical background information, supporting arguments, or side discussions should not be marked.

Anaphora: Sometimes, discussions involve anaphoric references to previously discussed aspects of the action item, such as "send me that". In this case, you should mark the utterance containing the anaphor and proceed backward in time to mark the utterance which contains the referent. However, some anaphors are more topical and reach very far back into discussion. If there are a number of intermediate utterances such that the original referent is not part of the same local subdialogue, you should not mark it.

Ownership and personal deixis: Usually, the name of a responsible person is not explicitly mentioned. You should therefore allow yourself to consider deictic references to individuals as sufficient for marking ownership utterances. This can come in the form of personal and possessive pronouns (e.g. "me", "you", "mine"), as well as possessive adjectives (e.g. "your"), as being an explicit communication of ownership information.

Writing the relevant sub-phrases

For each set of utterances in each dimension (ActionItemSubgroup), you should copy the directly relevant sub-phrases and place them in the NOMOS-description field of the utterance subgroup. Subphrases from different utterances should be separated by a pipe character ('|'). Subphrases from the same utterance should be separated by the ('#') character.

Agreement

Use the following guidelines for annotating the agreement dimension of an action item discussion:

Agreeing: Any utterance which communicates that a person is in agreement with some quality (or sometimes the entire definition) of the action item should be marked. Also, a lower-case "a" should be written into the comment field of the utterance. It is often hard to distinguish between agreement, acknowledgement, and backchannelling. Only mark those utterances which you believe truly communicate an agreement on the intentional level (i.e. the person is approving of a suggestion which is relevant to the action item, not just saying "I hear you" or "I understand you"). You will usually need to listen to the audio to determine this, rather than simply read the transcript. Note that an agreement utteranace might also be communicating "I hear you" or "I understand you", but this quality is not sufficient in itself to mark the utterance as an agreement.

Commitment: Any utterance which confirms commitment or impose it on someone else should also be marked. Also, a lower-case "c" should be written into the comment field of the utterance. Questions concerning commitment such as "Promise?" or "So you'll do it?" should not be marked (though the latter of these example would in fact be marked as an owner utterance), since they do not communicate a commitment - they only suggest or request it. Commands which impose commitment should be marked, such as "All right. You're doing it."