Updated Instruction Overview

Relationships should only be used within 1 sentence. Do not link labels between different sentences.
Relationship 'Caused by (Forårsaket av)' is changed to Is observed with (Er observert med) because causality cannot be determined.
Injection (injeksjon) and Infusion (infusjon) can be annoated using the Procedure category (Prosedyre) and injection site (innstikksted) can be annotated using the Anatomical location category (Anatomisk plassering)
Click for labels
- Procedure category (Prosedyre)
  - Injection (Injeksjon)
    - Intradermal injection (Intradermalt injeksjon)
    - Subcutaneous injection (Subkutant injeksjon)
    - Intramuscular injection (Intramuskulært injeksjon)
    - Intravenous injection (Intravenøst injeksjon)
    - Self-inflicted injection (Selvpåført injeksjon)
  - Infusion (Infusjon)
    - Subcutaneous infusion (Subkutant infusjon)
    - Intraosseous infusion (Intraossøs infusjon)
    - Intravenous infusion (Intravenøst infusjon)
- Anatomical location category (Anatomisk plassering)
  - Injection site (Innstikkssted)
Before: After:

The following categories for annotation have been changed:

Old Category	New Category	Category Description
~~Whole (Hel)~~	Annotated note (Annotert notat)	a label indicating a note has been annotated and used as a checklist of relevant content
~~Sensitivity (Sensitiv)~~	Identifier (Identifikator)	protected health information
~~Sign (Tegn)~~	Observation (Observasjon)	observable signs, such as patient signs or symptoms, lab results, and device-related observations
~~Location (Plassering)~~	Anatomical location (Anatomisk lokasjon)	anatomical location
~~Device (Enhet)~~	Medical device (Medisinsk utstyr)	different catheter types and parts
Procedure (Prosedyre)		procedures, interventions, activities, or procedural-related purposes

Summary of entity attributes to provide more information for the following categories:

Category or Subcategory	Entity Attribute	Options
Annotated note (Annotert notat)	• Has identifier (Har identifikator) • Has medical device malfunction (Har medisinsk utstyrsfeil) • Is catheter-related (Er kateterrelatert) • Is PVK-related (Er PVK relatert) • Is infection-related (Er infeksjonsrelatert) • Is infection-related (Er infeksjonsrelatert) • Is BSI-related (Er BSI relatert) • Is sepsis-related (Er sepsisrelatert)	checkmark = yes
Person (Person)	Is name (Er navn)	checkmark = yes
Body temperature (Kroppstemperatur)	Has body temperature range (Har kroppstemperaturområde)	• Hyperthermia (Hypertermi)• Normal (Normal)• Hypothermia (Hypotermi)
• Respiratory rate (Respirasjons frekvens) • Pulse (Puls) • Blood pressure (Blodtrykk) • C-reactive protein (CRP; C-reaktivt protein)	Has severity level (Har alvorlighetsgrad)	• High (Høy) • Normal (Normal) • Low (Lav)
Conciousness level (Bevissthetsnivå)	Has concious state of (Har bevisste tilstand av)	• Alert (Våken) • Confusion (Forvirring) • Verbally responsive (Verbal respons) • Painfully responsive (Smerte respons) • Unresponsive (Bevisstløshet)
Observation (Observasjon)	Observation said by (Observasjon sagt av)	• Health care provider (helsepersonell) • Patient (pasient) • Both (begge)
Observation (Observasjon)	Observation is diagnosis (Observasjon er diagnose)	checkmark = yes
Body part (Kroppsdel)	Left or right (Venstre eller høyre)	• Left (Venstre) • Right (Høyre)
Body part (Kroppsdel)	Front or back (Foran eller bak)	• Front (Foran) • Back (Bak)
Body part (Kroppsdel)	Distance location (Avstand plassering)	• Proximal (Proksimal) • Distal (Distal) • Medial (Medial) • Sideways (Sideveis) • Halfway (Halvveis)
Medical device (Medisinsk utstyr)	Is medical device failure (Er medisinsk utstyrsfeil)	checkmark = yes
Catheter (Kateter)	Is bandage (Er bandasje)	checkmark = yes
Peripheral intravenous catheter (PVK)	Has PIVC color (Har PVK farge)	• Purple (Lilla) • Yellow (Gul) • Blue (Blå) • Pink (Rosa) • Green (Grønn) • White (Hvit) • Gray (Grå) • Brown (Brun) • Orange (Oransje)
Procedure (Prosedyre)	Is incorrect procedure action (Er feil prosedyre handling)	checkmark = yes
• Observation (Observasjon) • Anatomical location (Anatomisk plassering) • Medical device (Medisinsk utstyr) • Procedure (Prosedyre)	Negation (Negasjon)	checkmark = this is not

Background

Of the 30 million patients who develop sepsis annually worldwide, over 5 million die. Sepsis a complex disease that can lead to tissue damage, organ failure, and death caused by a dysregulated host response to infection spread through the bloodstream. Bloodstream infections (BSIs) can enter the body through peripheral intravenous catheters (PVKs), which are the most commonly used invasive devices worldwide in hospitals. 80% of patients admitted to hospitals receive one or more PVKs.

Annotation is identifying and marking up text with labels to categorize different concepts. In addition to categorizing concepts, it is possible to link categories based on relationships. The annotated adverse event text samples from this study will be used to automatically identify risk factors and signs of PVK-related BSI to assist in preventing, identifying and treating sepsis.

Instructions

Log on to BRAT with your provided username and password in a Chrome browser.

Click for example
Annotate Norwegian anonymous synthetic adverse event notes:
- Annotate the whole note using the checklist for Annotated note (Annotert notat).
- Annotate parts of text using the guideline for the 6 categories:
  1. Identifier (Identifikator): protected health information
  2. Person (Person): individuals
  3. Observation (Observasjon): observed abnormality
  4. Anatomical location (Anatomisk lokasjon): anatomical location
  5. Medical device (Medisinsk utstyr): different catheter types and parts
  6. Procedure (Prosedyre): procedures, interventions, or activities
    
    Click for example

How to Annotate with BRAT

Using the Brat rapid annotation tool (BRAT) it is possible to:

Add and delete annotations for categories.

Click for example
Provide additional attribute information in the 'entity attributes' section if applicable.

Click for example
Add and delete relationships (Add relationships by clicking 1 label and dragging the arrow to another label; Delete by double clicking the relationship).

Click for example
Include notes in annotations.

Click for example

General Annotation Rules

Label all notes, even ones that seem irrelevant to PVK or sepsis.
- We need to give the machine learning model good examples and bad examples so it can learn "what is pvk and sepsis relevant" vs "what is not relevant".
Only annotate what is explicitly in the text, do not annotate inferences or speculations.
Label each word or phrase with only 1 label.
Click for example
- Incorrect
- Correct
Include nearby words that describe the labels and add 'entity attributes' to give additional label information whenever possible.

Click for example
Annotate a category or subcategory each time it is mentioned.
If a subcategory cannot be specified, chose the most general category level that describes the term or phrase.
Use the 'entity attribute' "Negation (Negasjon)" to indicate the opposite of something, failure, or if "it is not" a sign, location, device, or procedure.

Click for example
Link relationships between categories when possible to assist the machine in learning what the label is referring to:
- Who has the sign, location, device, or procedure? (Person har)
- What device does a procedure use? (Prosedyre bruker)
- What device or procedure is observered with a sign? (Er observert med)
- Where is the sign, device, or procedure located? (Lokalisert ved/på/i)
Label type of catheter care separately from the catheter type and link them using the relationship "Procedure uses (Prosedyre bruker)". This makes it possible to identify type of care given to a specific kind of catheter.
Click for labels and example
- Procedure category (Prosedyre)
  - Catheter procedure (Kateter prosedyre)
    - Catheter replacment (Kateter bytting)
    - Catheter removal (Kateter fjerning)
    - Catheter insertion (Kateter innstikking)
    - Catheter self-removal (Kateter selvseponering)
    - Catheter discontinuation (Kateter seponering)
    - Catheter rinsing (Kateter skylling)
- Medical device category (Medisinsk utstyr)
  - Catheter (Kateter)
    - Venous catheter (Venekateter)
    - Peripheral intravenous catheter (PIVC) (Perifert venekateter (PVK))
      
      Central venous catheter (Sentralt venekateter)
      
      Hemodialysis catheter (Hemodialysekateter)
      
      Midline catheter (Mideline kateter)
      
      Peripherally inserted central catheter (PICC) (Perifert innsatt sentalt kateter (PICC))
      
      Hickman catheter (Hickman kateter)
      
      Vascath catheter (VAS) (VAS-kateter (VAS))
    - Arterial catheter (Arteriekateter)
    - Urinary catheter (Blærekateter)
    - Epidural catheter (Epidural kateter)
    - Intraosseous cannula (Intraosseøs kanyle)
    - Subcutaneous catheter (Subkutant kateter)
Before: After:

Annotation Guidelines for Peripheral Intravenous Catheters related to Bloodstream Infections

Annotation is identifying and marking up text with labels to categorize different concepts. In addition to categorizing concepts, it is possible to link categories based on relationships. The following sections describe how to annotate the whole note and parts of notes using the six categories and draw relationship links between categories.

Annotation Rules

Annotated note Category (Annotert notat)

The whole note can be annotated by selecting the first word, "Hele_Notater", and checking off items listed in the 'entity attributes' which are true:

Has identifier (Har identifikator): Does the note contain sensitive information?
Has medical device malfunction (Har medisinsk utstyrsfeil): Is this note about a device malfunction?
Is catheter-related (Er kateterrelatert): Is this note related to catheters?
Is PVK-related (Er PVK relatert): Is this note related to peripheral intravenous catheters?
Is infection-related (Er infeksjonsrelatert): Is this note related to infections?
Is BSI-related (Er BSI relatert): Is this note related to bloodstream infections?
Is sepsis-related (Er sepsisrelatert): Is this note about sepsis?

Given the following example:

Unannotated "Annotated note" example

The result should be:

Annotated "Annotated note" example

Annotated Note (Annotert Notat) Annotation Rules

This should only be used to label the word "Hele_Notater" at the top of each note.
- Notes that are true for any of the questions listed under the Annotated Note Category (Annotert Notat) should have the corresponding attribute selected.

Identifier Category (Identifikator)

Protected health information is sensitive data that can identify a patient. These are the following subcategories:

Time (Tid)
Numeral identifier (Numerisk identifikator)
- Sample or test number (Prøve eller testnummer)
- Norwegian national identity number (Fødselsnummer)
- patient identification number in Doculive (PID)
- Account number (Kontonummer)
- Telephone number (Telefonnummer)
- Device number (Utstyrsnummer)
Geographic location identifier (Geografiske lokasjon identifikator)
Email (e-post)
Vehicle identifier (Kjøretøy identifikator)
Web URL (Nettadresse)
IP address (IP adresse)
Biometric identifier (Biometrisk identifikator)

* note: Name is part of the Person Category (Person).

Given the following example:

Unannotated "Identifier" example

The result should be:

Annotated "Identifier" example

Identifier Annotation Rules

All sensitive data should be annotated.
Dates and/or times should be categorized under Time (Tid).
Hospital wards/departments and hospital names (ex. St. Olav) should be categorized under Geographic location identifier (Geografiske lokasjon identifikator).

Person Category (Person)

Individuals are people mentioned in the notes. The individuals mentioned in the note should be annotated with a label based on their role in the 3 subcategories:

Patient (Pasient)
Health care provider (Helsepersonell)
Relative of the patient (Slektning)

Given the following example:

Unannotated "Person" example

The result should be:

Annotated "Person" example

Person Annotation Rules

Individual roles should be assigned to one of the subcategories.
Include names with their role and check Is name (Er navn) in the 'entity attributes' section.

Observation (Observasjon)

Certain signs can indicate a patient has an observed abnormal condition (such as an infection) or a device has a malfunction. The subcategories are as follows:

Descriptive sign or symptom (Beskrivende tegn eller symptom)
- Bleeding (Blødning)
- Bloody (Blodig)
- Phlebitis (Flebitis)
- Hardness (Hardhet)
- Headache (Hodepine)
- Swollen (Hoven)
- Infection (Infeksjon)
- Itchy (Kløende)
- Nausea (Kvalme)
- Necrosis (Nekrose)
- Purulence (Puss)
- Red (Rød)
- Pain (Smerte)
- Sepsis (Sepsis)
- Dizziness (Svimmelhet)
- Warm (Varm)
- Tenderness (Ømhet)
- Edema (Ødem)
Vital sign (Vitale livstegn)
- Blood pressure (Blodtrykk)
- Body temperature (Kroppstemperatur)
- Consciousness level (Bevissthetsnivå)
- Pulse (Puls)
- Respiratory rate (Respirasjons frekvens)
Neurological and physiological (Nevrologisk og fysiologisk)
- Mobility impairment (Bevegelseshemming)
- Psychosis (Psykose)
Wound (Sår)
- Laceration (Laserasjon)
- Abrasion (Skrubbsår)
- Incision (Snittsår)
- Puncture (Stikksår)
Lab result (Laboratorieresultat)
- Blood culture (Blodkultur)
- C-reactive protein (CRP; C-reaktivt protein)
- Leukocyte (Leukocytt)
Device observation (Utstyrsobservasjon)
- Moisture (Fukt)
- Leakage (Lekkasje)
- Liquid (Væske)

Given the following example:

Unannotated "Observation" example

The result should be:

Annotated "Observation" example

Observation (Observasjon) Annotation Rules

Annotate each observation separately and include adjectives. Use negation to indicate a sign is not present.
Identify the device or procedure observed with an observation using the "Is observed with (Er observert med)" relationship to link the observation to the device or procedure.

Include these 'entity attributes':

Subcategories	Entity Attribute	Options
Body temperature (Kroppstemperatur)	Has body temperature range (Har kroppstemperaturområde)	• Hyperthermia (Hypertermi) • Normal (Normal) • Hypothermia (Hypotermi)
• Respiratory rate (Respirasjons frekvens) • Pulse (Puls) • Blood pressure (Blodtrykk) • C-reactive protein (CRP; C-reaktivt protein)	Has severity level (Har alvorlighetsgrad)	• High (Høy) • Normal (Normal) • Low (Lav)
Conciousness level (Bevissthetsnivå)	Has concious state of (Har bevisste tilstand av)	• Alert (Våken) • Confusion (Forvirring) • Verbally responsive (Verbal respons) • Painfully responsive (Smerte respons) • Unresponsive (Bevisstløshet)
All from "Observation category (Observasjon)"	Observation said by (Observasjon sagt av)	• Health care provider (helsepersonell) • Patient (pasient) • Both (begge)
All from "Observation category (Observasjon)"	Observation is diagnosis (Observasjon er diagnose)	checkmark = yes, this is a diagnosis

Anatomical location Category (Anatomisk plassering)

Anatomical locations of devices and observations can identify the source of the issue. Also, anatomical insertion sites of catheters can be indicators of the type of catheter used.

Here is the list of subcategories:

Body part (Kroppsdel)
- Limb (Lem)
  - Arm (Arm)
    - Hand (Hånd)
    - Wrist (Håndledd)
    - Forearm (Underarm)
    - Elbow (Albue)
    - Upper arm (Overarm)
  - Leg (Bein)
    - Ankle (Ankel)
    - Foot (Fot)
    - Thigh (Lår)
    - Hip (Hofte)
- Head (Hode)
- Neck (Hals)
- Trunk (Torso)
  - Navel (Navle)
  - Back (Rygg)
Organ part (Organdel)
- Skin (Hud)
  - Subcutaneous (Subkutan)
- Circulartory system (Sirkulasjonssystem)
  - Vein (Ven)
  - Artery (Arterie)
Injection site (Innstikkssted)

Given the following example:

Unannotated "Anatomical location" exmaple

The result should be:

Annotated "Anatomical location" exmaple

Anatomical location Annotation Rules

Whenever it is explicitly stated, always provide additional information about the insertion site by filling in options in the 'entity attributes' section, such as which side of the body and distance:
- Left (Venstre)
- Right (Høyre)
- Front (Foran)
- Back (Bak)
- Proximal (Proksimal)
- Distal (Distal)
- Medial (Medial)
- Sideways (Sideveis)
- Halfway (Halvveis)

Medical device Category (Medisinsk utstyr)

Different catheters and parts of catheters must be identified so that machine learning classifiers can distinguish peripheral intravenous catheters (PVKs) from other catheters. Names and parts of catheter are provided here:

Catheter (Kateter)
- Venous catheter (Venekateter)
  - Peripheral intravenous catheter (PIVC) (Perifert venekateter (PVK))
  - Central venous catheter (Sentralt venekateter)
  - Hemodialysis catheter (Hemodialysekateter)
  - Midline catheter (Midline kateter)
  - Peripherally inserted central catheter (PICC) (Perifert innsatt sentralt kateter (PICC))
  - Hickman catheter (Hickman kateter)
  - Vascath catheter (VAS) (VAS-kateter (VAS))
- Arterial catheter (Arteriekateter)
- Urinary catheter (Blærekateter)
- Epidural catheter (Epidural kateter)
- Intraosseous cannula (Intraosseøs kanyle)
- Subcutaneous catheter (Subkutant kateter)
Pump (Pumpe)
Tubes (Rør)
Valve (Ventil)
- 3-way valve (Treveiskran)

Given the following example:

Unannotated "Medical device" example

The result should be:

Annotated "Medical device" example

Medical device Annotation Rules

Annotate catheters and other devices from the list.
If the observation was observed with a device not listed, then annotate it as the highest category "Medical device (Medisinsk utstyr)". Be sure to identify the device and draw a "Is observed with (Er observert med)" relationship to link the sign to device.
Catheter bandages should have the 'entity attribute' "Is bandage (Er bandasje)" selected.
For peripheral intravenous catheter (Perifert venekateter), it is possible to select the following colors for 'entity attribute':
- Purple (Lilla)
- Yellow (Gul)
- Blue (Blå)
- Pink (Rosa)
- Green (Grønn)
- White (Hvit)
- Gray (Grå)
- Brown (Brun)
- Orange (Oransje)
If context indicates the medical device or part has failed, it should have the 'entity attribute' Is medical device failure (Er medisinsk utstyrsfeil).

Procedure Category (Prosedyre)

Specific procedures and interventions will require the use of certain catheters even if they are not mentioned. Additionally, activities related to catheters can indicate the type of catheter used. Subcategories for the procedure category are:

Administration purpose (Administrasjonsformål)
- General IV (Generelt IV)
  - IV antibiotics (IV antibiotika)
  - IV chemo (IV cellegift)
  - IV medication (IV medisiner)
  - IV nutrient (IV næringsstoff)
  - IV painkiller (IV smertestillende)
  - IV fluid (IV væske)
- Blood test (Blodprøve)
- Blood transfusion (Blodverføring)
- Hemodialysis (Hemodialyse)
- Oral medication (Oral medisinering)
Administration way (Administrasjonsvei)
- Epidural use (Epidural bruk)
- IV use (IV bruk)
- Oral use (Oral bruk)
Arrived by Ambulance (Ankom med ambulanse)
Catheter procedure (Kateter prosedyre)
- Catheter replacement (Kateter bytting)
- Catheter removal (Kateter fjerning)
- Catheter insertion (Kateter innstikking)
- Catheter self-removal (Kateter selvseponering)
- Catheter discontinuation (Kateter seponering)
- Catheter rinsing (Kateter skylling)
Surgery (Kirurgi)
Injection (Injeksjon)
- Intradermal injection (Intradermalt injeksjon)
- Intramuscular injection (Intramuskulært injeksjon)
- Intravenous injection (Intravenøst injeksjon)
- Self-inflicted injection (Selvpåført injeksjon)
- Subcutaneous injection (Subkutant injeksjon)
Infusion (Infusjon)
- Intraosseous infusion (Intraossøs infusjon)
- Intravenous infusion (Intravenøst infusjon)
- Subcutaneous infusion (Subkutant infusjon)
Radiology (Radiologi)

Given the following example:

Unannotated "Procedure" example

The result should be:

Annotated "Procedure" example

Procedure Annotation Rules

Annotate the procedures, interventions, and activities related to catheters.
If the abnormal sign was observed with an action not listed, then annotate it as the highest category "Procedure (Prosedyre)". Be sure to identify the procedure and draw a "Is observed with (Er observert med)" relationship to link the sign to procedure.
Label all IV or oral medication names present.
Incorrect procedures should have the 'entity attribute' "Is incorrect procedure action (Er feil prosedyre handling)"

All Relationships

All Relationship Rules

It is alright to draw relationship links between different sentences within the same note.

Person has (Person har)

A person has an observed abnormal sign, anatomical location, device, and/or procedure, intervention, or activity.

Observation (Observasjon)
Anatomical location (Anatomisk plassering)
Medical device (Medisinsk utstyr)
Procedure (Prosedyre)

Click for example

Procedure uses (Prosedyre bruker)

The procedure uses a device.

Medical device (Medisinsk utstyr)

Click for example

Is observed with (Er observert med)

A sign is observed with a device or an event.

Medical device (Medisinsk utstyr)
Procedure (Prosedyre)

Click for example

Located nearby/on/at/in (Lokalisert ved/på/i)

The abnormal sign, device, and/or procedure is located nearby/on/in this anatomical location.

Observation (Observasjon)
Medical device (Medisinsk utstyr)
Procedure (Prosedyre)

Click for example

Given the following example:

Unannotated "Relationship" example

The result should be:

Annotated "Relationship" example

Adverse Event Guideline (version 5)

Last Updated: 15. December 2021 at 15:00

Updated Instruction Overview

Background

Instructions

How to Annotate with BRAT

General Annotation Rules