Adverse Event Guideline (version 4)

---

Overview of Updated Instructions

1. Label hospital ward or department as 'Geographic data (Geografiske data)'.

2. Label signs of cognitive function, confusion, psychosis, and emotional state when possible.

   Click for labels

   • Sign category (Tegn)
     • Cognitive function (Kognitiv funksjon)
       • Confusion (Forvirring)
       • Psychosis (Psykose)
       • Emotional state (Følelsesmessige tilstand)

3. Label type of catheter care separately from the catheter type and link them using the relationship "Procedure uses (Prosedyre bruker)". This makes it possible to identify type of care given to a specific kind of catheter.

   Click for labels and example

   • Procedure category (Prosedyre)
     • Catheter care (Kateter stell)
       • Insertion (Innleggelse)
       • Observation (Observasjon)
       • Rinse (Skylle)
       • Replace (Bytt)
       • Removal (Fjerning)
       • Discontinue (Seponert)
       • Self removal (Selvseponert)
   • Device category (Enhet)
     • Catheter (Kateter)
       • Venous catheter (Venekateter)
         • Hemodialysis catheter (Hemodialysekateter)
         • Central venous catheter (Sentralt venekateter)
         • Midline (Midline)
         • PICC (PICC)
         • Hickman (Hickman)
         • VAS (VAS)
         • Peripheral venous catheter (Perifert venekateter)
       • Arterial catheter (Arteriekateter)
       • Urinary catheter (Blærekateter)
       • Epidural (Epidural)

   

4. 'Subcutaneous infusions (Subkutane infusjoner)', 'injection site (innstikksted), or puncture site (stikksted)' should be labeled based on the type of injection (injeksjon) given using the Procedure category (Prosedyre) . Additionally, 'entity attribute' "Incorrect action (Feil handling)" is used to indicate an incorrectly given injection.

   Click for labels and example

   • Procedure category (Prosedyre)
     • Injection (Injeksjon)
       • Intradermal injection (Intradermalt injeksjon)
       • Subcutaneous injection (Subkutant injeksjon)
       • Intramuscular injection (Intramuskulært injeksjon)
       • Intravenous injection (Intravenøst injeksjon)

   Before: After:

5. Missing subcategories and corrections are in red below.

   Click for update summary

   • Location category (Plassering)
     • Leg (Bein)
       • Hip (Hofte)
     • Skin (Hud)
       • Surface (Overflate)
       • Subcutaneous (Subkutan)
     • Circulatory system (Sirkulasjonssystem)
       • Vein (Ven)
       • Artery (Arterie)
   • Device category (Enhet)
     • Catheter (Kateter)
       • Venous catheter (Venekateter)
         • Peripheral venous catheter (Perifert venekateter)
       • Arterial catheter (Arteriekateter)
   • Procedure category (Prosedyre)
     • Injection (Injeksjon)
       • Intradermal injection (Intradermalt injeksjon)
       • Subcutaneous injection (Subkutant injeksjon)
       • Intramuscular injection (Intramuskulært injeksjon)
       • Intravenous injection (Intravenøst injeksjon)
     • Device malfunction (Systemfeil)
       • Moisture (Fukt)
       • Leakage (Lekkasje)
       • Liquid (Væske)
   • All Relationships : each relationship has an example now for clarity.

6. Missing 'entity attributes' are in red below.

   Click for update summary

   • Whole category (Hel)
     • 'Sepsis (Sepsis)' is now an option
   • Device category (Enhet)
     • 'Bandage (Bandasje)' is now an option for 'Catheter (Kateter)' and it's subcategories
     • Updated color options for 'Peripheral venous catheter (Perifert venekateter)' to include: 'Pink (Rosa)', 'Purple (Lilla)', 'White (Hvit)'
   • Procedure category (Prosedyre)
     • 'Incorrect action (Feil handling)' is now an option for 'Injection (Injeksjon)' and it's subcategories

---

Background

Of the 30 million patients who develop sepsis annually worldwide, over 5 million die. Sepsis a complex disease that can lead to tissue damage, organ failure, and death caused by a dysregulated host response to infection spread through the bloodstream. Bloodstream infections (BSIs) can enter the body through peripheral venous catheters (PVKs), which are the most commonly used invasive devices worldwide in hospitals. 80% of patients admitted to hospitals receive one or more PVKs.

Annotation is identifying and marking up text with labels to categorize different concepts. In addition to categorizing concepts, it is possible to link categories based on relationships. The annotated adverse event text samples from this study will be used to automatically identify risk factors and signs of PVK-related BSI to assist in preventing, identifying and treating sepsis.

---

Instructions

1. Log on to BRAT with your provided username and password in a Chrome browser.

   Click for example

   

2. Annotate Norwegian anonymous synthetic adverse event notes:
   • Annotate the whole note using the checklist for Whole (Hel).
   • Annotate parts of text using the guideline for the 6 categories:
     1. Sensitivity (Sensitiv): protected health information
     2. Person (Person): individuals
     3. Sign (Tegn): observed abnormality
     4. Location (Plassering): anatomical location
     5. Device (Enhet): different catheter types and parts
     6. Procedure (Prosedyre): procedures, interventions, or activities
        Click for example

        

---

How to Annotate with BRAT

Using the Brat rapid annotation tool (BRAT) it is possible to:

• Add and delete annotations for categories.
  Click for example

  
• Provide additional attribute information in the 'entity attributes' section if applicable.
  Click for example

  
• Add and delete relationships (Add relationships by clicking 1 label and dragging the arrow to another label; Delete by double clicking the relationship).
  Click for example

  
  
• Include notes in annotations.
  Click for example

  

---

General Annotation Rules

1. Label all notes, even ones that seem irrelevant to PVK or sepsis.
   • We need to give the machine learning model good examples and bad examples so it can learn "what is pvk and sepsis relevant" vs "what is not relevant".
2. Only annotate what is explicitly in the text, do not annotate inferences or speculations.
3. Label each word or phrase with only 1 label.
   Click for example

   • Incorrect
     
   • Correct
     
4. Include nearby words that describe the labels and add 'entity attributes' to give additional label information whenever possible.
   Click for example

   
5. Annotate a category or subcategory each time it is mentioned.
6. If a subcategory cannot be specified, chose the most general category level that describes the term or phrase.
7. Use the 'entity attribute' "Negation (Negasjon)" to indicate the opposite of something, failure, or if "it is not" a sign, location, device, or procedure.
   Click for example

   
8. Link relationships between categories when possible to assist the machine in learning what the label is referring to:
   • Who has the sign, location, device, or procedure? (Person har)
   • What device does a procedure use? (Prosedyre bruker)
   • What device or procedure caused a sign? (Forårsaket av)
   • Where is the sign, device, or procedure located? (Lokalisert ved/på/i)

---

Annotation Guidelines for Peripheral Venous Catheters related to Bloodstream Infections

Annotation is identifying and marking up text with labels to categorize different concepts. In addition to categorizing concepts, it is possible to link categories based on relationships. The following sections describe how to annotate the whole note and parts of notes using the six categories and draw relationship links between categories.

---

Annotation Rules

---

Whole Category (Hel)

The whole note can be annotated by selecting the first word, "Hele_Notater", in the note ("Hel") and checking off items listed in the 'entity attributes' which are true:

• Sensitivity (Sensitiv): Does the note contain sensitive information?
• Device malfunction (Feil på enheten): Is this note about a device malfunction?
• Catheter-related (Kateterrelatert): Is this note related to catheters?
• PVK related (PVK relatert): Is this note related to peripheral intravenous catheters?
• Infection related (Infeksjonsrelatert): Is this note related to infections?
• BSI related (BSI relatert): Is this note related to bloodstream infections?
• Sepsis (Sepsis): Is this note about sepsis?

Given the following example:

The result should be:

Whole Annotation Rules

1. This should only be used to label the word "Hele_Notater" at the top of each note.
   • Notes that are true for any of the questions listed under the Whole (Hel) should have the corresponding attribute selected.

---

Sensitivity Category (Sensitiv)

Protected health information is sensitive data that can identify a patient. These are the following subcategories:

• Time and date (Tid og dato)
• Numbers (Nummer)
  • Sample or test number (Prøve- eller testnummer)
  • Norwegian birth number (Fødselsnummer)
  • Personal identifier of patient in Doculive (PID)
  • Account number (Kontonummer)
  • Telephone number (Telefonnummer)
  • Device number (Enhetsnummer)
• Geographic data (Geografiske data)
• Email (e-post)
• Vehicle identifier (Kjøretøy identifikator)
• Web URL (Nettadresse)
• IP address (IP adresse)
• Biometric identifier (Biometrisk identifikator)

* note: Name is part of the Person Category (Person).

Given the following example:

The result should be:

Sensitivity Annotation Rules

1. All sensitive data should be annotated.
2. Dates and times should be categorized under "Time and date (Tid og dato)".
3. Hospital wards/departments and hospital names (ex. St. Olav) should be categorized under "Geographic data (geografiske data)".

---

Person Category (Person)

Individuals are people mentioned in the notes. The individuals mentioned in the note should be annotated with a label based on their role in the 3 subcategories:

• Patient (Pasient)
• Healthcare provider (Helsepersonell)
• Relative of the patient (Slektning)

Given the following example:

The result should be:

Person Annotation Rules

1. Individual roles should be assigned to one of the subcategories.
2. Include names with their role and check "Name (Nav)" in the 'entity attributes' section.

---

Sign Category (Tegn)

Certain signs can indicate a patient has an observed abnormal condition (such as an infection). The subcategories are as follows:

• Pain (Smerte)
• Tenderness (Ømhet)
• Pus (Pus)
• Purulence (Purulens)
• Red (Rød)
• Warm (Varm)
• Swollen (Hoven)
• Edema (Ødem)
• Phlebitis (Flebitis)
• Itchy (Kløende)
• Trauma (Traume)
  • Bleeding (Blødning)
  • Injury (Skade)
• Mobility impairment (Bevegelseshemming)
• Sepsis (Sepsis)
• Respiratory rate (Respirasjons frekvens)
• Pulse (Puls)
• Blood pressure (Blodtrykk)
• Nausea (Kvalme)
• Headache (Hodepine)
• Dizziness (Svimmel)
• Cognitive function (Kognitiv funksjon)
  • Confusion (Forvirring)
  • Psychosis (Psykose)
  • Emotional state (Følelsesmessige tilstand)
• Body temp (Kroppstemp)
• C-reactive protein (CRP; C-reaktivt protein)
• Necrosis (Nekrose)
• Infection (Infeksjon)

Given the following example:

The result should be:

Sign Annotation Rules

1. Annotate each sign separately and include adjectives. Use negation to indicate a sign is not present.
2. Annotate signs and symptoms clearly related and not related to infections. Be sure to identify the cause of the sign using a relationship to link the sign to the cause.
3. Vital signs identified should have additional severity information added in the 'entity attributes' section in BRAT whenever possible to indicate if the condition is normal or abnormal.
   • For Body temp (Kroppstemp), it is possible to select the following from the dropdown menu:
     • Normal (Normal)
     • Hyperthermia (Hypertermi)
     • Hypothermia (Hypotermi)
       
   • The Low (Lav), Normal (Normal), or High (Høy) severity is available for:
     • Respiratory rate (Respirasjons frekvens)
     • Pulse (Puls)
     • Blood pressure (Blodtrykk)
     • Cognitive function (Kognitiv funksjon)
     • C-reactive protein (CRP; C-reaktivt protein)

---

Location Category (Plassering)

Anatomical locations of devices and events can identify the source of the issue. Also, anatomical insertion sites of catheters can be indicators of the type of catheter used. Here is the list of subcategories:

• Limb (Limb)
  • Arm (Arm)
    • Hand (Hånd)
    • Wrist (Håndledd)
    • Forearm (Underarm)
    • Elbow (Albue)
    • Upper arm (Overarm)
  • Leg (Bein)
    • Ankle (Ankel)
    • Foot (Fot)
    • Thigh (Lår)
    • Hip (Hofte)
• Head (Hode)
• Neck (Hals)
• Trunk (Torso)
  • Navel (Navle)
  • Back (Rygg)
• Skin (Hud)
  • Surface (Overflate)
  • Subcutaneous (Subkutan)
• Circulatory system (Sirkulasjonssystem)
  • Vein (Ven)
  • Artery (Arterie)

Given the following example:

The result should be:

Location Annotation Rules

1. Whenever it is explicitly stated, always provide additional information about the insertion site by filling in options in the 'entity attributes' section, such as which side of the body and distance:

   • Left (Venstre)
   • Right (Høyre)
   • Front (Fran)
   • Back (Bak)
   • Proximal (Proksimal)
   • Distal (Distal)
   • Medial (Medial)
   • Sideways (Sideveis)
   • Halfway (Halvveis)

---

Device Category (Enhet)

Different catheters and parts of catheters must be identified so that machine learning classifiers can distinguish peripheral intravenous catheters (PVKs) from other catheters. Names and parts of catheter are provided here:

• Catheter (Kateter)
  • Venous catheter (Venekateter)
    • Hemodialysis catheter (Hemodialysekateter)
    • Central venous catheter (Sentralt venekateter)
    • Midline (Midline)
    • PICC (PICC)
    • Hickman (Hickman)
    • VAS (VAS)
    • Peripheral venous catheter (Perifert venekateter)
  • Arterial catheter (Arteriekateter)
  • Urinary catheter (Blærekateter)
  • Epidural (Epidural)
• Device Parts (Enhetsdeler)
  • Pump (Pumpe)
  • Tubes (Rør)
  • Valves (Ventiler)

Given the following example:

The result should be:

Device Annotation Rules

1. Annotate catheters and other devices from the list.
2. If the abnormal sign was caused by a device not listed, then annotate it as the highest category "Device (Enhet)". Be sure to identify the device and draw a "Caused by (Forårsaket av)" relationship to link the sign to device.
3. Device parts (Enhetsdeler) not related to catheters should have the 'entity attribute' "Not related to catheters (Ikke relatert til kateter)" selected.
4. Catheter bandages should have the 'entity attribute' "Bandage (Bandasje)" selected.
5. For peripheral venous catheter (Perifert venekateter), it is possible to select the following colors for 'entity attribute':
   • Purple (Lilla)
   • Yellow (Gul)
   • Blue (Blå)
   • Pink (Rosa)
   • Green (Grønn)
   • White (Hvit)
   • Gray (Grå)
   • Brown (Brun)
   • Orange (Orange)

---

Procedure Category (Prosedyre)

Specific procedures and interventions will require the use of certain catheters even if they are not mentioned. Additionally, activities related to catheters can indicate the type of catheter used. Device malfunction is included so it can be separated from catheter related procedures and interventions. Subcategories for the procedure category are:

• Blood transfusion (Blodverføring)
• Blood test (Blodprøve)
• Hemodialysis (Hemodialyse)
• Pain pump (Smertepumpe)
• Surgery (Kirurgi)
• Administration (Administrasjon)
  • General IV (Generelt IV)
    • IV medication (IV medisiner)
      • IV antibiotics (IV antibiotika)
      • IV chemo (IV cellegift)
    • IV fluids (IV væsker)
      • IV nutrient (IV næringsstoff)
      • IV liquid (IV væske)
  • Oral medication (oral medisinering)
• Arrived by Ambulance (ankom med ambulanse)
• Catheter care (Kateter stell)
  • Insertion (Innleggelse)
  • Rinse (Skylle)
  • Replace (Bytt)
  • Removal (Fjerning)
  • Discontinue (Seponert)
  • Self removal (Selvseponert)
• Injection (Injeksjon)
  • Intradermal injection (Intradermalt injeksjon)
  • Subcutaneous injection (Subkutant injeksjon)
  • Intramuscular injection (Intramuskulært injeksjon)
  • Intravenous injection (Intravenøst injeksjon)
• Device Malfunction (Systemfeil)
  • Moisture (Fukt)
  • Leakage (Lekkasje)
  • Liquid (Væske)

Given the following example:

The result should be:

Procedure Annotation Rules

1. Annotate the procedures, interventions, and activities related to catheters.
2. If the abnormal sign was caused by an action not listed, then annotate it as the highest category "Procedure (Prosedyre)". Be sure to identify the procedure and draw a "Caused by (Forårsaket av)" relationship to link the sign to procedure.
3. Label all IV or oral medicine names present.
4. Incorrectly given injections should have the 'entity attribute' "Incorrect action (Feil handling)" selected.

---

All Relationships

All Relationship Rules

1. It is alright to draw relationship links between different sentences within the same note.

Person has (Person har)

A person has an observed abnormal sign, anatomical location, device, and/or procedure, intervention, or activity.

• Sign (Tegn)
• Location (Plassering)
• Device (Enhet)
• Procedure (Prosedyre)

Click for example

Procedure uses (Prosedyre bruker)

The procedure uses a device.

• Device (Enhet)

Click for example

Caused by (Forårsaket av)

A sign is caused by a device or an event.

• Device (Enhet)
• Procedure (Prosedyre)

Click for example

Located nearby/on/in (Lokalisert ved/på/i)

The abnormal sign, device, and/or procedure is located nearby/on/in this anatomical location.

• Sign (Tegn)
• Device (Enhet)
• Procedure (Prosedyre)

Click for example

Given the following example:

The result should be: