Adverse Event Guideline (version 2)

---

Overview of Updated Instructions

1. Label all notes, even ones that seem irrelevant to PVK or sepsis.
   • We need to give the machine learning model good examples and bad examples so it can learn "what is pvk and sepsis relevant" vs "what is not relevant".
2. Label each word or phrase with 1 label.
   Click for example

   • Incorrect
     
   • Correct
     
3. Link labels with relationships when applicable to assist the machine in learning what the label is referring to:
   • Who has the sign, location, device, or procedure? ( Person har )
   • What device does a procedure use? ( Prosedyre bruker )
   • What device or procedure caused a sign? ( Forårsaket av )
   • Where is the sign, device, or procedure located? ( Ligger på )

4. Include nearby words that describe the labels and add 'entity attributes' to give additional label information whenever possible.
   

   Click for example

   

5. Add "negation" to "entity attribute" if "it is not" a sign, location, device, or procedure.

   Click for example

   

6. The label Whole (Hel) should only be used to label the word "Hele_Notater" at the top of each note. Notes that are true for any of the questions listed under the Whole Category (Hel) should have this label.

7. Categories revisions are in red below.
   Click for update summary

   • Changed "Date (Dato)" to "Time and Date (Tid og dato)" in Sensitive category (Sensitiv) to clarify times and/or dates should be labeled.
   • For names, use Person category (Person) and check the entity attribute "Name (Nav)".
   • Sign category (Tegn) should include adjectives about signs.
   • Location category (Plassering) example shows possible label for "innstikkstedet".
   • Device category (Enhet) example shows label for a part of catheters.
   • Procedure category (Prosedyre) example shows labeles for infusion and medications.
   • Removed relationship option Person is (Person er)

---

Background

Of the 30 million patients who develop sepsis annually worldwide, over 5 million die. Sepsis a complex disease that can lead to tissue damage, organ failure, and death caused by a dysregulated host response to infection spread through the bloodstream. Bloodstream infections (BSIs) can enter the body through peripheral venous catheters (PVKs), which are the most commonly used invasive devices worldwide in hospitals. 80% of patients admitted to hospitals receive one or more PVKs.

Annotation is identifying and marking up text with labels to categorize different concepts. In addition to categorizing concepts, it is possible to link categories based on relationships. The annotated adverse event text samples from this study will be used to automatically identify risk factors and signs of PVK-related BSI to assist in preventing, identifying and treating sepsis.

---

Instructions

1. Log on to BRAT with your provided username and password in a Chrome browser.

   Click for example

   

2. Annotate Norwegian anonymous synthetic adverse event notes:
   • Annotate the whole note using the checklist for Whole (Hel).
   • Annotate parts of text using the guideline for the 6 categories:
     1. Sensitivity (Sensitiv): protected health information
     2. Person (Person): individuals
     3. Sign (Tegn): observed abnormality
     4. Location (Plassering): anatomical location
     5. Device (Enhet): different catheter types and parts
     6. Procedure (Prosedyre): procedures, interventions, or activities
        Click for example

        

---

How to Annotate with BRAT

Using the Brat rapid annotation tool (BRAT) it is possible to :

• Add and delete annotations for categories.
  Click for example

  
• Provide additional attribute information in the "entity attributes" section if applicable.
  Click for example

  
• Add and delete relationships (Add relationships by clicking 1 label and dragging the arrow to another label; Delete by double clicking the relationship).
  Click for example

  
  
• Include notes in annotations.
  Click for example

  

---

General Annotation Rules

1. Only annotate what is explicitly in the text, do not annotate inferences or speculations.
2. Annotate a category or subcategory each time it is mentioned.
3. Link relationships between categories when possible.
4. Provide additional category information from the drop-down selection when possible.
5. If a subcategory cannot be specified, chose the most general category level that describes the term or phrase.

---

Annotation Guidelines for Peripheral Venous Catheters related to Bloodstream Infections

Annotation is identifying and marking up text with labels to categorize different concepts. In addition to categorizing concepts, it is possible to link categories based on relationships. The following sections describe how to annotate the whole note and parts of notes using the six categories and draw relationship links between categories.

---

Annotation Rules

---

Whole Category (Hel)

The whole note can be annotated by selecting the first word, "Hele_Notater", in the note ("Hel") and checking off items listed in the "entity attributes" which are true:

• Sensitivity (Sensitiv): Does the note contain sensitive information?
• Device malfunction (Feil på enheten): Is this note about a device malfunction?
• Catheter-related (Kateterrelatert): Is this note related to catheters?
• PVK related (PVK relatert): Is this note related to peripheral intravenous catheters?
• Infection related (Infeksjonsrelatert): Is this note related to infections?
• BSI related (BSI relatert): Is this note related to bloodstream infections?

Given the following example:

The result should be:

---

Sensitivity Category (Sensitiv)

Protected health information is sensitive data that can identify a patient. These are the following subcategories:

• Time and date (Tid og dato)
• Numbers (Nummer)
  • Sample or test number (Prøve- eller testnummer)
  • Norwegian birth number (fødselsnummer)
  • Personal identifier of patient in Doculive (PID)
  • Account number (kontonummer)
  • Telephone number (telefonnummer)
  • Device number (enhetsnummer)
• Geographic data (geografiske data)
• Email (e-post)
• Vehicle identifier (kjøretøy identifikator)
• Web URL (nettadresse)
• IP address (IP adresse)
• Biometric identifier (biometrisk identifikator)

* note: Name is part of the Person Category (Person).

Given the following example:

The result should be:

Sensitivity Annotation Rules

1. All sensitive data should be annotated.
2. Dates and times should be categorized under "Time and date (Tid og dato)".

---

Person Category (Person)

Individuals are people mentioned in the notes. The individuals mentioned in the note should be annotated with a label based on their role in the 3 subcategories:

• Patient (Pasient)
• Healthcare provider (Helsepersonell)
• Relative of the patient (Slektning)

Given the following example:

The result should be:

Person Annotation Rules

1. Individual roles should be assigned to one of the subcategories.
2. Include names with their role and check "Nav" in the "entity attributes" section.

---

Sign Category (Tegn)

Certain signs can indicate a patient has an observed abnormal condition, such as an infection. The subcategories are as follows:

• Pain (Smerte)
• Tenderness (Ømhet)
• Pus (Pus)
• Purulence (Purulens)
• Red (Rød)
• Warm (Varm)
• Swollen (Hoven)
• Edema (Ødem)
• Phlebitis (Flebitis)
• Sepsis (Sepsis)
• Respiratory rate (Respirasjons frekvens)
• Pulse (Puls)
• Blood pressure (Blodtrykk)
• Nausea (Kvalme)
• Mental state (Mental tilstand)
• Body temp (Kroppstemp)
• C-reactive protein (CRP; C-reaktivt protein)
• Necrosis (Nekrose)
• Infection (Infeksjon)

Given the following example:

The result should be:

Sign Annotation Rules

1. Annotate each sign separately and include adjectives. Use negation to indicate a sign is not present.
2. Annotate signs and symptoms clearly related and not related to infections. Be sure to identify the cause of the sign using a relationship to link the sign to the cause.
3. Vital signs identified should have additional severity information added in the "entity attributes" section in BRAT whenever possible to indicate if the condition is normal or abnormal.
   • For body temp (Kroppstemp), it is possible to select the following from the dropdown menu:
     • Normal (Normal)
     • Hyperthermia (Hypertermi)
     • Hypothermia (Hypotermi)
       
   • The Low (Lav), Normal (Normal), or High (Høy) severity is available for:
     • Respiratory rate (Respirasjons frekvens)
     • Pulse (Puls)
     • Blood pressure (Blodtrykk)
     • Mental state (Mental tilstand)
     • C-reactive protein (CRP; C-reaktivt protein)

---

Location Category (Plassering)

Anatomical locations of devices and events can identify the source of the issue. Also, anatomical insertion sites of catheters can be indicators of the type of catheter used. Here is the list of subcategories:

• Limb (Limb)
  • Arm (Arm)
    • Hand (Hånd)
    • Wrist (Håndledd)
    • Forearm (Underarm)
    • Elbow (Albue)
    • Upper arm (Overarm)
  • Leg (Bein)
    • Ankle (Ankel)
    • Foot (Fot)
    • Thigh (Lår)
• Head (Hode)
• Neck (Hals)
• Trunk (Torso)
  • Navel (Navle)
• Skin (Hud)
  • Surface (Overflate)
  • Subcutaneous (Subkutan)
• Elsewhere (Andre steder)
  • Vein (Ven)
  • Artery (Arterie)

Given the following example:

The result should be:

Location Annotation Rules

1. Whenever it is explicitly stated, always provide additional information about the insertion site by filling in options in the "entity attributes" section, such as which side of the body and distance:

   • Left (Venstre)
   • Right (Høyre)
   • Front (Fran)
   • Back (Bak)
   • Proximal (Proksimal)
   • Distal (Distal)
   • Medial (Medial)
   • Sideways (Sideveis)
   • Halfway (Halvveis)

---

Device Category (Enhet)

Different catheters and parts of catheters must be identified so that machine learning classifiers can distinguish peripheral intravenous catheters (PVKs) from other catheters. Names and parts of catheter are provided here:

• Catheter (Kateter)
  • Venous catheter (Venekateter)
    • Hemodialysis catheter (hemodialysekateter)
    • Central venous catheter (Sentralt venekateter)
    • Midline (Midline)
    • PICC (PICC)
    • Hickman (Hickman)
    • VAS (VAS)
  • Peripheral venous catheter (Perifert venekateter)
  • Urinary catheter (Blærekateter)
• Device Parts (enhetsdeler)
  • Pump (Pumpe)
  • Tubes (Rør)
  • Valves (Ventiler)

Given the following example:

The result should be:

Device Annotation Rules

1. Annotate catheters and other devices from the list.
2. If the abnormal sign was caused by a device not listed, then annotate it as the highest category "Device (Enhet)". Be sure to identify the device and draw a "Caused by (Forårsaket av)" relationship to link the sign to device.

---

Procedure Category (Prosedyre)

Specific procedures and interventions will require the use of certain catheters even if they are not mentioned. Additionally, activities related to catheters can indicate the type of catheter used. Subcategories for the procedure category are:

• Blood transfusion (Blodverføring)
• Blood test (Blodprøve)
• Hemodialysis (Hemodialyse)
• Epidural (Epidural)
• Pain pump (Smertepumpe)
• Surgery (Kirurgi)
• Administration (Administrasjon)
  • General IV (Generelt IV)
    • IV medication (IV medisiner)
      • IV antibiotics (IV antibiotika)
      • IV chemo (IV cellegift)
    • IV fluids (IV væsker)
      • IV nutrient (IV næringsstoff)
      • IV liquid (IV væske)
  • Oral medication (oral medisinering)
• Arrived by Ambulance (ankom med ambulanse)

Given the following example:

The result should be:

Procedure Annotation Rules

1. Annotate the procedures, interventions, and activities related to catheters.
2. If the abnormal sign was caused by an action not listed, then annotate it as the highest category "Procedure (Prosedyre)". Be sure to identify the procedure and draw a "Caused by (Forårsaket av)" relationship to link the sign to procedure.

---

All Relationships

All Relationship Rules

1. It is alright to draw relationship links between different sentences within the same note.

Person has (Person har)

A person has an observed abnormal sign, anatomical location, device, and/or procedure, intervention, or activity.

• Sign (Tegn)
• Location (Plassering)
• Device (Enhet)
• Procedure (Prosedyre)

Procedure uses (Prosedyre bruker)

The procedure uses a device.

• Device (Enhet)

Caused by (Forårsaket av)

A sign is caused by a device or an event.

• Device (Enhet)
• Procedure (Prosedyre)

Located at (Ligger på)

The abnormal sign, device, and/or procedure is located at this anatomical location.

• Sign (Tegn)
• Device (Enhet)
• Procedure (Prosedyre)

Given the following example:

The result should be: