From 6e03fab118e50f8e0278c8644526eb3f7fb7389b Mon Sep 17 00:00:00 2001 From: Maxim Moinat Date: Thu, 23 Feb 2023 08:21:52 +0100 Subject: [PATCH] initial tooling support page --- docs/cdm54ToolingSupport.RMD | 110 ++ docs/cdm54ToolingSupport.html | 1462 ++++++++++++++++++++++++ rmd/_site.yml | 2 + rmd/cdm54.Rmd | 4 +- rmd/cdm54ToolingSupport.RMD | 110 ++ rmd/cdm54ToolingSupport_full_table.Rmd | 362 ++++++ 6 files changed, 2049 insertions(+), 1 deletion(-) create mode 100644 docs/cdm54ToolingSupport.RMD create mode 100644 docs/cdm54ToolingSupport.html create mode 100644 rmd/cdm54ToolingSupport.RMD create mode 100644 rmd/cdm54ToolingSupport_full_table.Rmd diff --git a/docs/cdm54ToolingSupport.RMD b/docs/cdm54ToolingSupport.RMD new file mode 100644 index 0000000..606658b --- /dev/null +++ b/docs/cdm54ToolingSupport.RMD @@ -0,0 +1,110 @@ +--- +title: "**OMOP CDM v5.4 Detailed Tooling Support**" +output: + html_document: + toc: TRUE + toc_float: TRUE +--- + +# Introduction +This tables below contain an overview of which standard OHDSI tools make use of which OMOP CDM fields. +The goal is to inform ETL developers, tooling developers and CDM extensions. + +- For ETL developers it helps to have guidance on which fieds to prioritise in the mapping. Most value will be gained from populating fields support across the OHDSI tooling. +- For OHDSI tooling developers, this page provides insight in the gaps of support and can drive future development efforts. +- For CDM extenstions, it helps to known what it means for an OMOP CDM table/field to be part of the standard. In other words: what OHDSI tooling do we at least expect to support the new extensions? + +Currently four OHDSI tools have been evaluated: DataQualityDashboard, Achilles, Atlas (Data Sources and Cohort creation) and Feature Extraction. + +## Criteria +Tool | Supports CDM Field if | Link to resource used +--- | --- | --- +DataQualityDashboard | Part of Field Level checks | https://github.com/OHDSI/DataQualityDashboard/blob/main/inst/csv/OMOP_CDMv5.4_Field_Level.csv +Achilles | Covered by at least one Achilles analysis | https://github.com/OHDSI/Achilles/blob/main/inst/csv/achilles/achilles_analysis_details.csv +Atlas Data Sources | A statistic based on the field is shown in a 'Data Sources' visualisation | https://atlas-demo.ohdsi.org/ +Atlas Cohort | Used in te Atlas User Interface for cohort definition criteria (directly, or via 'Add attribute') | https://atlas-demo.ohdsi.org/ +Feature Extraction | Used in one of the Feature Extraction analyses | https://github.com/OHDSI/FeatureExtraction/blob/main/inst/csv/ + +General criteria: +- `r emo::ji("white heavy check mark")` if the field essential for OMOP CDM definition (Primary Key, Foreign Key) e.g. person_id not explicitly used, but essential. (if the PK is marked as False, it typically means the whole table is not used in the tool e.g. `_source_value` fields that are used for traceability in ETL) +- `r emo::ji("warning")` if field is used by the tool, but not in a meaningful way. e.g. `provider_id` in Achilles only checked for a valid foreign key to the provider table. + +# Tooling Support for OMOP fields + **Abbrevations** |   +--- | --- +**PK** | Primary Key +**SV** | Source Value (for data quality / etl validation) +**BC** | Backwards Compatibility, to support CDM + + + + + + + + + + + + +OMOP CDM v5.4 Detailed Tooling Support + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ + + +
+
+
+
+
+ +
+ + + + + + + + + +
+

Introduction

+

This tables below contain an overview of which standard OHDSI tools +make use of which OMOP CDM fields. The goal is to inform ETL developers, +tooling developers and CDM extensions.

+
    +
  • For ETL developers it helps to have guidance on which fieds to +prioritise in the mapping. Most value will be gained from populating +fields support across the OHDSI tooling.
    • +
  • For OHDSI tooling developers, this page provides insight in the gaps +of support and can drive future development efforts.
    • +
  • For CDM extenstions, it helps to known what it means for an OMOP CDM +table/field to be part of the standard. In other words: what OHDSI +tooling do we at least expect to support the new extensions?
    • +
+

Currently four OHDSI tools have been evaluated: DataQualityDashboard, +Achilles, Atlas (Data Sources and Cohort creation) and Feature +Extraction.

+
+

Criteria

+ +++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ToolSupports CDM Field ifLink to resource used
DataQualityDashboardPart of Field Level checkshttps://github.com/OHDSI/DataQualityDashboard/blob/main/inst/csv/OMOP_CDMv5.4_Field_Level.csv
AchillesCovered by at least one Achilles analysishttps://github.com/OHDSI/Achilles/blob/main/inst/csv/achilles/achilles_analysis_details.csv
Atlas Data SourcesA statistic based on the field is shown in a ‘Data Sources’ +visualisationhttps://atlas-demo.ohdsi.org/
Atlas CohortUsed in te Atlas User Interface for cohort definition criteria +(directly, or via ‘Add attribute’)https://atlas-demo.ohdsi.org/
Feature ExtractionUsed in one of the Feature Extraction analyseshttps://github.com/OHDSI/FeatureExtraction/blob/main/inst/csv/
+

General criteria: - ✅ if the field essential for OMOP CDM definition +(Primary Key, Foreign Key) e.g. person_id not explicitly used, but +essential. (if the PK is marked as False, it typically means the whole +table is not used in the tool e.g. _source_value fields +that are used for traceability in ETL) - ⚠️ if field is used by the +tool, but not in a meaningful way. e.g. provider_id in +Achilles only checked for a valid foreign key to the provider table.

+
+
+
+

Tooling Support for OMOP fields

+ ++++ + + + + + + + + + + + + + + + + + + + + + + + + +
Abbrevations 
PKPrimary Key
SVSource Value (for data quality / etl validation)
BCBackwards Compatibility, to support CDM <v5.4
FCForwards Compatibility, to easy support for CDM v6 in the +future.
+
+

Person

+ ++++++++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
cdmTableNamecdmFieldNameSpecial FieldsDQD (v1.0)Achilles (v1.7)Atlas Cohort (v2.10)Atlas Cohort (v2.12)Atlas Data Sources (v2.12)Feature Extraction (v3.2)Comment
PERSONperson_idPK✅✅✅✅✅✅
PERSONgender_concept_id✅✅✅✅✅✅
PERSONyear_of_birth✅✅✅✅✅✅
PERSONmonth_of_birth✅✅❌❌❌❌
PERSONday_of_birth✅❌❌❌❌❌
PERSONbirth_datetimeFC✅❌❌❌❌❌
PERSONrace_concept_id✅✅✅✅✅✅
PERSONethnicity_concept_id✅✅✅✅✅✅
PERSONlocation_id✅⚠️❌❌❌❌Achilles only does FK check
PERSONprovider_id✅⚠️❌❌❌❌Achilles only does FK check
PERSONcare_site_id✅⚠️❌❌❌❌Achilles only does FK check
PERSONperson_source_valueSV✅❌❌❌❌❌
PERSONgender_source_valueSV✅❌❌❌❌❌
PERSONgender_source_concept_id✅❌❌❌❌❌
PERSONrace_source_valueSV✅❌❌❌❌❌
PERSONrace_source_concept_id✅❌❌❌❌❌
PERSONethnicity_source_valueSV✅❌❌❌❌❌
PERSONethnicity_source_concept_id✅❌❌❌❌❌
+
+
+

Observation Period

+ ++++++++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
cdmTableNamecdmFieldNameSpecial FieldsDQD (v1.0)Achilles (v1.7)Atlas Cohort (v2.10)Atlas Cohort (v2.12)Atlas Data Sources (v2.12)Feature Extraction (v3.2)Comment
OBSERVATION_PERIODobservation_period_idPK✅✅✅✅✅✅
OBSERVATION_PERIODperson_idPid✅✅✅✅✅✅
OBSERVATION_PERIODobservation_period_start_date✅✅✅✅✅✅
OBSERVATION_PERIODobservation_period_end_date✅✅✅✅✅✅
OBSERVATION_PERIODperiod_type_concept_id✅✅✅✅❌❌
+
+
+

Visit Occurrence

+ ++++++++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
cdmTableNamecdmFieldNameSpecial FieldsDQD (v1.0)Achilles (v1.7)Atlas Cohort (v2.10)Atlas Cohort (v2.12)Atlas Data Sources (v2.12)Feature Extraction (v3.2)Comment
VISIT_OCCURRENCEvisit_occurrence_idPK✅✅✅✅✅✅
VISIT_OCCURRENCEperson_idPid✅✅✅✅✅✅
VISIT_OCCURRENCEvisit_concept_id✅✅✅✅✅✅
VISIT_OCCURRENCEvisit_source_concept_id✅✅✅✅❌❌
VISIT_OCCURRENCEvisit_source_valueSV✅⚠️❌❌❌❌
VISIT_OCCURRENCEvisit_start_date✅✅✅✅✅✅Achilles check 1900
VISIT_OCCURRENCEvisit_start_datetimeFC✅❌❌❌❌❌
VISIT_OCCURRENCEvisit_end_date✅✅✅✅❌✅
VISIT_OCCURRENCEvisit_end_datetimeFC✅❌❌❌❌❌
VISIT_OCCURRENCEvisit_type_concept_id✅✅✅✅❌❌
VISIT_OCCURRENCEprovider_id✅❌✅✅❌❌Atlas uses provider.specialty_concept_id
VISIT_OCCURRENCEcare_site_id✅⚠️✅✅❌❌Achilles only does FK check, Atlas uses +care_site.place_of_service_concept_id
VISIT_OCCURRENCEadmitted_from_concept_id✅❌❌❌❌❌
VISIT_OCCURRENCEadmitted_from_source_valueSV✅⚠️❌❌❌❌Achilles check 1900
VISIT_OCCURRENCEdischarged_to_concept_id✅❌❌❌❌❌
VISIT_OCCURRENCEdischarged_to_source_valueSV✅⚠️❌❌❌❌Achilles check 1900
VISIT_OCCURRENCEpreceding_visit_occurrence_id✅❌❌❌❌❌
+
+
+

Episode

+ ++++++++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
cdmTableNamecdmFieldNameSpecial FieldsDQD (v1.0)Achilles (v1.7)Atlas Cohort (v2.10)Atlas Cohort (v2.12)Atlas Data Sources (v2.12)Feature Extraction (v3.2)Comment
EPISODEepisode_idPK✅❌❌❌❌❌
EPISODEperson_idPid✅❌❌❌❌❌
EPISODEepisode_concept_id✅❌❌❌❌❌
EPISODEepisode_start_date✅❌❌❌❌❌
EPISODEepisode_start_datetimeFC✅❌❌❌❌❌
EPISODEepisode_end_date✅❌❌❌❌❌
EPISODEepisode_end_datetimeFC✅❌❌❌❌❌
EPISODEepisode_parent_id✅❌❌❌❌❌
EPISODEepisode_number✅❌❌❌❌❌
EPISODEepisode_object_concept_id✅❌❌❌❌❌
EPISODEepisode_type_concept_id✅❌❌❌❌❌
EPISODEepisode_source_valueSV✅❌❌❌❌❌
EPISODEepisode_source_concept_id✅❌❌❌❌❌
+

This was an effort by the CDM Working Group in 2022. *Credits: Clair +Blacketer, Maxim Moinat, Nitin Park

+
+
+ + + +
+
+ +
+ + + + + + + + + + + + + + + + diff --git a/rmd/_site.yml b/rmd/_site.yml index 57284e6..4f314ae 100644 --- a/rmd/_site.yml +++ b/rmd/_site.yml @@ -50,6 +50,8 @@ navbar: href: cdm54Changes.html - text: "Entity Relationships" href: cdm54erd.html + - text: "Detailed tooling support per CDM field" + href: "cdm54ToolingSupport.html" - text: "CDM Proposals" icon: fa-plus-square menu: diff --git a/rmd/cdm54.Rmd b/rmd/cdm54.Rmd index 87e5b38..ee09ce5 100644 --- a/rmd/cdm54.Rmd +++ b/rmd/cdm54.Rmd @@ -23,7 +23,9 @@ This is the specification document for the OMOP Common Data Model, v5.4. **This ### Current Support for CDM v5.4 -The table below details which OHDSI tools support CDM v5.4. There are two levels of support: legacy support means that the tool supports all tables and fields that were present in CDM v5.3 and feature support indicates that the tool supports any new tables and fields in CDM v5.4 that were not present in CDM v5.3. A green check `r emo::ji("white heavy check mark")` indicates that the support level for the listed tool is in place, has been tested, and released. A warning sign `r emo::ji("warning")` indicates that the support level for the listed tool has been initiated but has not yet been tested and released. +The table below details which OHDSI tools support CDM v5.4. There are two levels of support: legacy support means that the tool supports all tables and fields that were present in CDM v5.3 and feature support indicates that the tool supports any new tables and fields in CDM v5.4 that were not present in CDM v5.3. A green check `r emo::ji("white heavy check mark")` indicates that the support level for the listed tool is in place, has been tested, and released. A warning sign `r emo::ji("warning")` indicates that the support level for the listed tool has been initiated but has not yet been tested and released. + +A more detailed support for CDM v5.4 is available in [cdm54ToolingSupport.html]. For select tooling (currently DataQualityDashboard, Achilles, Atlas and FeatureExtraction), it shows what OMOP fields are actually covered by that particular tool. For example, none of the fields in the `note` table are referenced in Atlas or FeatureExtraction.
diff --git a/rmd/cdm54ToolingSupport.RMD b/rmd/cdm54ToolingSupport.RMD new file mode 100644 index 0000000..606658b --- /dev/null +++ b/rmd/cdm54ToolingSupport.RMD @@ -0,0 +1,110 @@ +--- +title: "**OMOP CDM v5.4 Detailed Tooling Support**" +output: + html_document: + toc: TRUE + toc_float: TRUE +--- + +# Introduction +This tables below contain an overview of which standard OHDSI tools make use of which OMOP CDM fields. +The goal is to inform ETL developers, tooling developers and CDM extensions. + +- For ETL developers it helps to have guidance on which fieds to prioritise in the mapping. Most value will be gained from populating fields support across the OHDSI tooling. +- For OHDSI tooling developers, this page provides insight in the gaps of support and can drive future development efforts. +- For CDM extenstions, it helps to known what it means for an OMOP CDM table/field to be part of the standard. In other words: what OHDSI tooling do we at least expect to support the new extensions? + +Currently four OHDSI tools have been evaluated: DataQualityDashboard, Achilles, Atlas (Data Sources and Cohort creation) and Feature Extraction. + +## Criteria +Tool | Supports CDM Field if | Link to resource used +--- | --- | --- +DataQualityDashboard | Part of Field Level checks | https://github.com/OHDSI/DataQualityDashboard/blob/main/inst/csv/OMOP_CDMv5.4_Field_Level.csv +Achilles | Covered by at least one Achilles analysis | https://github.com/OHDSI/Achilles/blob/main/inst/csv/achilles/achilles_analysis_details.csv +Atlas Data Sources | A statistic based on the field is shown in a 'Data Sources' visualisation | https://atlas-demo.ohdsi.org/ +Atlas Cohort | Used in te Atlas User Interface for cohort definition criteria (directly, or via 'Add attribute') | https://atlas-demo.ohdsi.org/ +Feature Extraction | Used in one of the Feature Extraction analyses | https://github.com/OHDSI/FeatureExtraction/blob/main/inst/csv/ + +General criteria: +- `r emo::ji("white heavy check mark")` if the field essential for OMOP CDM definition (Primary Key, Foreign Key) e.g. person_id not explicitly used, but essential. (if the PK is marked as False, it typically means the whole table is not used in the tool e.g. `_source_value` fields that are used for traceability in ETL) +- `r emo::ji("warning")` if field is used by the tool, but not in a meaningful way. e.g. `provider_id` in Achilles only checked for a valid foreign key to the provider table. + +# Tooling Support for OMOP fields + **Abbrevations** |   +--- | --- +**PK** | Primary Key +**SV** | Source Value (for data quality / etl validation) +**BC** | Backwards Compatibility, to support CDM