From cf5c366cc4ec49a93fe4cb9d6a174121e43ec6cd Mon Sep 17 00:00:00 2001 From: Clair Blacketer Date: Tue, 16 Apr 2024 16:13:00 -0400 Subject: [PATCH] Fixes to markdown --- docs/cdm54ToolingSupport.html | 1462 +++++++++++++++++++++++++++++++++ docs/customConcepts.html | 40 +- rmd/customConcepts.Rmd | 4 + 3 files changed, 1490 insertions(+), 16 deletions(-) create mode 100644 docs/cdm54ToolingSupport.html diff --git a/docs/cdm54ToolingSupport.html b/docs/cdm54ToolingSupport.html new file mode 100644 index 0000000..b563f70 --- /dev/null +++ b/docs/cdm54ToolingSupport.html @@ -0,0 +1,1462 @@ + + + + + + + + + + + + + +OMOP CDM v5.4 Detailed Tooling Support + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + +
+
+
+
+
+ +
+ + + + + + + + + +
+

Introduction

+

This tables below contain an overview of which standard OHDSI tools +make use of which OMOP CDM fields. The goal is to inform ETL developers, +tooling developers and CDM extensions.

+
    +
  • For ETL developers it helps to have guidance on which fields to +prioritise in the mapping. Most value will be gained from populating +fields support across the OHDSI tooling.
    • +
  • For OHDSI tooling developers, this page provides insight in the gaps +of support and can drive future development efforts.
    • +
  • For CDM extensions, it helps to known what it means for an OMOP CDM +table/field to be part of the standard. In other words: what OHDSI +tooling do we at least expect to support the new extensions?
    • +
+

Currently four OHDSI tools have been evaluated: DataQualityDashboard, +Achilles, Atlas (Data Sources and Cohort creation) and Feature +Extraction.

+
+

Criteria

+ +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ToolSupports CDM Field ifLink to resource used
DataQualityDashboardPart of Field Level checkshttps://github.com/OHDSI/DataQualityDashboard/blob/main/inst/csv/OMOP_CDMv5.4_Field_Level.csv
AchillesCovered by at least one Achilles analysishttps://github.com/OHDSI/Achilles/blob/main/inst/csv/achilles/achilles_analysis_details.csv
Atlas Data SourcesA statistic based on the field is shown in a ‘Data Sources’ +visualisationhttps://atlas-demo.ohdsi.org/
Atlas CohortUsed in te Atlas User Interface for cohort definition criteria +(directly, or via ‘Add attribute’)https://atlas-demo.ohdsi.org/
Feature ExtractionUsed in one of the Feature Extraction analyseshttps://github.com/OHDSI/FeatureExtraction/blob/main/inst/csv/
+

General criteria: - ✔️ if the field essential for OMOP CDM definition +(Primary Key, Foreign Key) e.g. person_id not explicitly used, but +essential. (if the PK is marked as False, it typically means the whole +table is not used in the tool e.g. _source_value fields +that are used for traceability in ETL) - ❗ if field is used by the +tool, but not in a meaningful way. e.g. provider_id in +Achilles only checked for a valid foreign key to the provider table.

+
+
+
+

Tooling Support for OMOP fields

+ ++++ + + + + + + + + + + + + + + + + + + + + + + + + +
Abbreviations 
PKPrimary Key
SVSource Value (for data quality / etl validation)
BCBackwards Compatibility, to support CDM <v5.4
FCForwards Compatibility, to easy support for CDM v6 in the +future.
+
+

Person

+ ++++++++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
cdmTableNamecdmFieldNameSpecial FieldsDQD (v1.0)Achilles (v1.7)Atlas Cohort (v2.10)Atlas Cohort (v2.12)Atlas Data Sources (v2.12)Feature Extraction (v3.2)Comment
PERSONperson_idPK✔️✔️✔️✔️✔️✔️
PERSONgender_concept_id✔️✔️✔️✔️✔️✔️
PERSONyear_of_birth✔️✔️✔️✔️✔️✔️
PERSONmonth_of_birth✔️✔️❗❗❗❗
PERSONday_of_birth✔️❗❗❗❗❗
PERSONbirth_datetimeFC✔️❗❗❗❗❗
PERSONrace_concept_id✔️✔️✔️✔️✔️✔️
PERSONethnicity_concept_id✔️✔️✔️✔️✔️✔️
PERSONlocation_id✔️❗❗❗❗❗Achilles only does FK check
PERSONprovider_id✔️❗❗❗❗❗Achilles only does FK check
PERSONcare_site_id✔️❗❗❗❗❗Achilles only does FK check
PERSONperson_source_valueSV✔️❗❗❗❗❗
PERSONgender_source_valueSV✔️❗❗❗❗❗
PERSONgender_source_concept_id✔️❗❗❗❗❗
PERSONrace_source_valueSV✔️❗❗❗❗❗
PERSONrace_source_concept_id✔️❗❗❗❗❗
PERSONethnicity_source_valueSV✔️❗❗❗❗❗
PERSONethnicity_source_concept_id✔️❗❗❗❗❗
+
+
+

Observation Period

+ ++++++++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
cdmTableNamecdmFieldNameSpecial FieldsDQD (v1.0)Achilles (v1.7)Atlas Cohort (v2.10)Atlas Cohort (v2.12)Atlas Data Sources (v2.12)Feature Extraction (v3.2)Comment
OBSERVATION_PERIODobservation_period_idPK✔️✔️✔️✔️✔️✔️
OBSERVATION_PERIODperson_idPid✔️✔️✔️✔️✔️✔️
OBSERVATION_PERIODobservation_period_start_date✔️✔️✔️✔️✔️✔️
OBSERVATION_PERIODobservation_period_end_date✔️✔️✔️✔️✔️✔️
OBSERVATION_PERIODperiod_type_concept_id✔️✔️✔️✔️❗❗
+
+
+

Visit Occurrence

+ ++++++++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
cdmTableNamecdmFieldNameSpecial FieldsDQD (v1.0)Achilles (v1.7)Atlas Cohort (v2.10)Atlas Cohort (v2.12)Atlas Data Sources (v2.12)Feature Extraction (v3.2)Comment
VISIT_OCCURRENCEvisit_occurrence_idPK✔️✔️✔️✔️✔️✔️
VISIT_OCCURRENCEperson_idPid✔️✔️✔️✔️✔️✔️
VISIT_OCCURRENCEvisit_concept_id✔️✔️✔️✔️✔️✔️
VISIT_OCCURRENCEvisit_source_concept_id✔️✔️✔️✔️❗❗
VISIT_OCCURRENCEvisit_source_valueSV✔️❗❗❗❗❗
VISIT_OCCURRENCEvisit_start_date✔️✔️✔️✔️✔️✔️Achilles check 1900
VISIT_OCCURRENCEvisit_start_datetimeFC✔️❗❗❗❗❗
VISIT_OCCURRENCEvisit_end_date✔️✔️✔️✔️❗✔️
VISIT_OCCURRENCEvisit_end_datetimeFC✔️❗❗❗❗❗
VISIT_OCCURRENCEvisit_type_concept_id✔️✔️✔️✔️❗❗
VISIT_OCCURRENCEprovider_id✔️❗✔️✔️❗❗Atlas uses provider.specialty_concept_id
VISIT_OCCURRENCEcare_site_id✔️❗✔️✔️❗❗Achilles only does FK check, Atlas uses +care_site.place_of_service_concept_id
VISIT_OCCURRENCEadmitted_from_concept_id✔️❗❗❗❗❗
VISIT_OCCURRENCEadmitted_from_source_valueSV✔️❗❗❗❗❗Achilles check 1900
VISIT_OCCURRENCEdischarged_to_concept_id✔️❗❗❗❗❗
VISIT_OCCURRENCEdischarged_to_source_valueSV✔️❗❗❗❗❗Achilles check 1900
VISIT_OCCURRENCEpreceding_visit_occurrence_id✔️❗❗❗❗❗
+
+
+

Episode

+ ++++++++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
cdmTableNamecdmFieldNameSpecial FieldsDQD (v1.0)Achilles (v1.7)Atlas Cohort (v2.10)Atlas Cohort (v2.12)Atlas Data Sources (v2.12)Feature Extraction (v3.2)Comment
EPISODEepisode_idPK✔️❗❗❗❗❗
EPISODEperson_idPid✔️❗❗❗❗❗
EPISODEepisode_concept_id✔️❗❗❗❗❗
EPISODEepisode_start_date✔️❗❗❗❗❗
EPISODEepisode_start_datetimeFC✔️❗❗❗❗❗
EPISODEepisode_end_date✔️❗❗❗❗❗
EPISODEepisode_end_datetimeFC✔️❗❗❗❗❗
EPISODEepisode_parent_id✔️❗❗❗❗❗
EPISODEepisode_number✔️❗❗❗❗❗
EPISODEepisode_object_concept_id✔️❗❗❗❗❗
EPISODEepisode_type_concept_id✔️❗❗❗❗❗
EPISODEepisode_source_valueSV✔️❗❗❗❗❗
EPISODEepisode_source_concept_id✔️❗❗❗❗❗
+

This was an effort by the CDM Working Group in 2022. *Credits: Clair +Blacketer, Maxim Moinat, Nitin Park

+
+
+ + + +
+
+ +
+ + + + + + + + + + + + + + + + diff --git a/docs/customConcepts.html b/docs/customConcepts.html index 23d775c..0e4b4ce 100644 --- a/docs/customConcepts.html +++ b/docs/customConcepts.html @@ -433,16 +433,21 @@ Custom concepts are concepts that are not part of the OMOP vocabularies, and are only used in your institution. There are two main reasons to define custom concepts in your local OMOP CDM vocabulary. The first is that they are available in your local Atlas instance, which has several -use cases: - When viewing a standard concept, you can see which custom -concepts are mapped to it. This allows you to better understand what the -standard concept represents in your institution. - You can search for a -custom concept and find which standard concepts it is mapped to, to -include in your standard concept set. - For studies only using your -local data, you can define cohorts using custom concepts (through ‘Add -attribute’->‘Add … Source Concept’). The second reason is using the -custom concepts in your ETL. By creating both the custom concept, and -the ‘Maps to’ relationship (example below), we can use this in the same -way as mapping other source vocabularies.

+use cases:

+ +

The second reason is using the custom concepts in your ETL. By +creating both the custom concept, and the ‘Maps to’ relationship +(example below), we can use this in the same way as mapping other source +vocabularies.

Custom concepts are only defined locally. These cannot be used for network research. Therefore it remains very important to map to standard concepts.

@@ -499,13 +504,16 @@ to’ concept should be a standard concept.
  • Officially, concept_hierarchy is only for standard concepts. However, if you local use case requires this (e.g. for selection of descendants of custom concepts), the custom concepts can be -added into their own, isolated, hierarchy. ## Example In this example, -we will add one custom concept for the ‘DHD Diagnose Thesaurus’. This is -a Dutch vocabulary, which is not part of the OMOP vocabularies. We will -add the concept ‘diabetes mellitus type 1’. This concept has a mapping -to the standard concept ‘Diabetes mellitus type 1 (disorder)’, -concept_id 3341872.
    • +added into their own, isolated, hierarchy. + +
    +

    Example

    +

    In this example, we will add one custom concept for the ‘DHD Diagnose +Thesaurus’. This is a Dutch vocabulary, which is not part of the OMOP +vocabularies. We will add the concept ‘diabetes mellitus type 1’. This +concept has a mapping to the standard concept ‘Diabetes mellitus type 1 +(disorder)’, concept_id 3341872.

    After creating these records, we can use the custom concept in our ETL to populate the condition_source_concept_id field.

    diff --git a/rmd/customConcepts.Rmd b/rmd/customConcepts.Rmd index 0e1eddb..4f94856 100644 --- a/rmd/customConcepts.Rmd +++ b/rmd/customConcepts.Rmd @@ -11,9 +11,11 @@ The Themis Working Group convened on October 6th and December 7th 2023 to discus While the OMOP vocabularies are very comprehensive, it is not always possible to use concepts existing in the OMOP vocabularies. For example, when using a vocabulary that is only used in your institution or having custom defined variables. In these cases, custom concepts can be used. Custom concepts are concepts that are not part of the OMOP vocabularies, and are only used in your institution. There are two main reasons to define custom concepts in your local OMOP CDM vocabulary. The first is that they are available in your local Atlas instance, which has several use cases: + - When viewing a standard concept, you can see which custom concepts are mapped to it. This allows you to better understand what the standard concept represents in your institution. - You can search for a custom concept and find which standard concepts it is mapped to, to include in your standard concept set. - For studies only using your local data, you can define cohorts using custom concepts (through 'Add attribute'->'Add ... Source Concept'). + The second reason is using the custom concepts in your ETL. By creating both the custom concept, and the 'Maps to' relationship (example below), we can use this in the same way as mapping other source vocabularies. **Custom concepts are only defined locally. These cannot be used for network research. Therefore it remains very important to map to standard concepts.** @@ -36,7 +38,9 @@ In addition, it is recommended to follow these suggestions: - In the new vocabulary record, the `vocabulary_concept_id` can be set to 0, as this is often not used in the OMOP CDM. - Create mappings between custom concepts and standard concepts. This can be done by adding rows to the `concept_relationship` table, with the `Maps to` relation. The reverse relation, `Mapped from`, should also be added. This allows for easy navigation between custom and standard concepts2. The 'mapped to' concept should be a standard concept. - Officially, `concept_hierarchy` is only for standard concepts. However, if you local use case requires this (e.g. for selection of descendants of custom concepts), the custom concepts can be added into their own, isolated, hierarchy. + ## Example + In this example, we will add one custom concept for the 'DHD Diagnose Thesaurus'. This is a Dutch vocabulary, which is not part of the OMOP vocabularies. We will add the concept 'diabetes mellitus type 1'. This concept has a mapping to the standard concept 'Diabetes mellitus type 1 (disorder)', concept_id 3341872. After creating these records, we can use the custom concept in our ETL to populate the `condition_source_concept_id` field.