Merge branch 'v5.4' into v5.4

This commit is contained in:
Alan Andryc 2021-08-19 13:30:50 -04:00 committed by GitHub
commit d73a0f426d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 1318 additions and 543 deletions

View File

@ -0,0 +1,603 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc" />
<meta http-equiv="X-UA-Compatible" content="IE=EDGE" />
<title>Indices, Primary Keys and Foreign Key Constraints</title>
<script src="site_libs/jquery-1.11.3/jquery.min.js"></script>
<meta name="viewport" content="width=device-width, initial-scale=1" />
<link href="site_libs/bootstrap-3.3.5/css/cosmo.min.css" rel="stylesheet" />
<script src="site_libs/bootstrap-3.3.5/js/bootstrap.min.js"></script>
<script src="site_libs/bootstrap-3.3.5/shim/html5shiv.min.js"></script>
<script src="site_libs/bootstrap-3.3.5/shim/respond.min.js"></script>
<style>h1 {font-size: 34px;}
h1.title {font-size: 38px;}
h2 {font-size: 30px;}
h3 {font-size: 24px;}
h4 {font-size: 18px;}
h5 {font-size: 16px;}
h6 {font-size: 12px;}
code {color: inherit; background-color: rgba(0, 0, 0, 0.04);}
pre:not([class]) { background-color: white }</style>
<script src="site_libs/jqueryui-1.11.4/jquery-ui.min.js"></script>
<link href="site_libs/tocify-1.9.1/jquery.tocify.css" rel="stylesheet" />
<script src="site_libs/tocify-1.9.1/jquery.tocify.js"></script>
<script src="site_libs/navigation-1.1/tabsets.js"></script>
<link href="site_libs/highlightjs-9.12.0/default.css" rel="stylesheet" />
<script src="site_libs/highlightjs-9.12.0/highlight.js"></script>
<link href="site_libs/font-awesome-5.1.0/css/all.css" rel="stylesheet" />
<link href="site_libs/font-awesome-5.1.0/css/v4-shims.css" rel="stylesheet" />
<link rel='shortcut icon' type='image/x-icon' href='favicon.ico' />
<style type="text/css">
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
span.underline{text-decoration: underline;}
div.column{display: inline-block; vertical-align: top; width: 50%;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
</style>
<style type="text/css">code{white-space: pre;}</style>
<script type="text/javascript">
if (window.hljs) {
hljs.configure({languages: []});
hljs.initHighlightingOnLoad();
if (document.readyState && document.readyState === "complete") {
window.setTimeout(function() { hljs.initHighlighting(); }, 0);
}
}
</script>
<link rel="stylesheet" href="style.css" type="text/css" />
<style type = "text/css">
.main-container {
max-width: 940px;
margin-left: auto;
margin-right: auto;
}
img {
max-width:100%;
}
.tabbed-pane {
padding-top: 12px;
}
.html-widget {
margin-bottom: 20px;
}
button.code-folding-btn:focus {
outline: none;
}
summary {
display: list-item;
}
pre code {
padding: 0;
}
</style>
<style type="text/css">
.dropdown-submenu {
position: relative;
}
.dropdown-submenu>.dropdown-menu {
top: 0;
left: 100%;
margin-top: -6px;
margin-left: -1px;
border-radius: 0 6px 6px 6px;
}
.dropdown-submenu:hover>.dropdown-menu {
display: block;
}
.dropdown-submenu>a:after {
display: block;
content: " ";
float: right;
width: 0;
height: 0;
border-color: transparent;
border-style: solid;
border-width: 5px 0 5px 5px;
border-left-color: #cccccc;
margin-top: 5px;
margin-right: -10px;
}
.dropdown-submenu:hover>a:after {
border-left-color: #adb5bd;
}
.dropdown-submenu.pull-left {
float: none;
}
.dropdown-submenu.pull-left>.dropdown-menu {
left: -100%;
margin-left: 10px;
border-radius: 6px 0 6px 6px;
}
</style>
<script type="text/javascript">
// manage active state of menu based on current page
$(document).ready(function () {
// active menu anchor
href = window.location.pathname
href = href.substr(href.lastIndexOf('/') + 1)
if (href === "")
href = "index.html";
var menuAnchor = $('a[href="' + href + '"]');
// mark it active
menuAnchor.tab('show');
// if it's got a parent navbar menu mark it active as well
menuAnchor.closest('li.dropdown').addClass('active');
// Navbar adjustments
var navHeight = $(".navbar").first().height() + 15;
var style = document.createElement('style');
var pt = "padding-top: " + navHeight + "px; ";
var mt = "margin-top: -" + navHeight + "px; ";
var css = "";
// offset scroll position for anchor links (for fixed navbar)
for (var i = 1; i <= 6; i++) {
css += ".section h" + i + "{ " + pt + mt + "}\n";
}
style.innerHTML = "body {" + pt + "padding-bottom: 40px; }\n" + css;
document.head.appendChild(style);
});
</script>
<!-- tabsets -->
<style type="text/css">
.tabset-dropdown > .nav-tabs {
display: inline-table;
max-height: 500px;
min-height: 44px;
overflow-y: auto;
border: 1px solid #ddd;
border-radius: 4px;
}
.tabset-dropdown > .nav-tabs > li.active:before {
content: "";
font-family: 'Glyphicons Halflings';
display: inline-block;
padding: 10px;
border-right: 1px solid #ddd;
}
.tabset-dropdown > .nav-tabs.nav-tabs-open > li.active:before {
content: "&#xe258;";
border: none;
}
.tabset-dropdown > .nav-tabs.nav-tabs-open:before {
content: "";
font-family: 'Glyphicons Halflings';
display: inline-block;
padding: 10px;
border-right: 1px solid #ddd;
}
.tabset-dropdown > .nav-tabs > li.active {
display: block;
}
.tabset-dropdown > .nav-tabs > li > a,
.tabset-dropdown > .nav-tabs > li > a:focus,
.tabset-dropdown > .nav-tabs > li > a:hover {
border: none;
display: inline-block;
border-radius: 4px;
background-color: transparent;
}
.tabset-dropdown > .nav-tabs.nav-tabs-open > li {
display: block;
float: none;
}
.tabset-dropdown > .nav-tabs > li {
display: none;
}
</style>
<!-- code folding -->
<style type="text/css">
#TOC {
margin: 25px 0px 20px 0px;
}
@media (max-width: 768px) {
#TOC {
position: relative;
width: 100%;
}
}
@media print {
.toc-content {
/* see https://github.com/w3c/csswg-drafts/issues/4434 */
float: right;
}
}
.toc-content {
padding-left: 30px;
padding-right: 40px;
}
div.main-container {
max-width: 1200px;
}
div.tocify {
width: 20%;
max-width: 260px;
max-height: 85%;
}
@media (min-width: 768px) and (max-width: 991px) {
div.tocify {
width: 25%;
}
}
@media (max-width: 767px) {
div.tocify {
width: 100%;
max-width: none;
}
}
.tocify ul, .tocify li {
line-height: 20px;
}
.tocify-subheader .tocify-item {
font-size: 0.90em;
}
.tocify .list-group-item {
border-radius: 0px;
}
</style>
</head>
<body>
<div class="container-fluid main-container">
<!-- setup 3col/9col grid for toc_float and main content -->
<div class="row">
<div class="col-xs-12 col-sm-4 col-md-3">
<div id="TOC" class="tocify">
</div>
</div>
<div class="toc-content col-xs-12 col-sm-8 col-md-9">
<div class="navbar navbar-default navbar-fixed-top" role="navigation">
<div class="container">
<div class="navbar-header">
<button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="index.html"><div><img src="ohdsi16x16.png"></img> OMOP Common Data Model </div></a>
</div>
<div id="navbar" class="navbar-collapse collapse">
<ul class="nav navbar-nav">
<li>
<a href="index.html">
<span class="fa fa-home"></span>
</a>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
<span class="fa fa-landmark"></span>
Background
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="background.html">Model Background</a>
</li>
<li>
<a href="vocabulary.html">How the Vocabulary is Built</a>
</li>
</ul>
</li>
<li>
<a href="dataModelConventions.html">
<span class="fa fa-list-alt"></span>
Conventions
</a>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
<span class="fa fa-history"></span>
CDM Versions
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="cdm30.html">CDM v3.0</a>
</li>
<li>
<a href="cdm531.html">CDM v5.3.1</a>
</li>
<li>
<a href="cdm531.html">CDM v5.4</a>
</li>
<li>
<a href="cdm60.html">CDM v6.0</a>
</li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
<span class="fa fa-plus-square"></span>
Proposals
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="reviewProposals.html">Under Review</a>
</li>
<li class="dropdown-submenu">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">Accepted</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="oncology.html">Oncology CDM Proposal</a>
</li>
<li>
<a href="https://github.com/OHDSI/CommonDataModel/issues/252">Region_concept_id</a>
</li>
<li>
<a href="https://github.com/OHDSI/CommonDataModel/issues/264">Units in Device Table</a>
</li>
</ul>
</li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
<span class="fa fa-question"></span>
Help
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="faq.html">FAQ</a>
</li>
<li>
<a href="contribute.html">Ask a Question</a>
</li>
</ul>
</li>
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-expanded="false">
<span class="fa fa-wrench"></span>
How to
<span class="caret"></span>
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="sqlScripts.html">SQL Scripts</a>
</li>
<li>
<a href="download.html">Download the DDL</a>
</li>
<li>
<a href="drug_dose.html">Calculate Drug Dose</a>
</li>
</ul>
</li>
</ul>
<ul class="nav navbar-nav navbar-right">
<li>
<a href="https://github.com/OHDSI/CommonDataModel">
<span class="fa fa-github fa-lg"></span>
</a>
</li>
</ul>
</div><!--/.nav-collapse -->
</div><!--/.container -->
</div><!--/.navbar -->
<div id="header">
<h1 class="title toc-ignore">Indices, Primary Keys and Foreign Key Constraints</h1>
</div>
<div id="to-do" class="section level3">
<h3>To Do</h3>
<ul>
<li>Need something for hashing</li>
<li>What about constraints that arent foreign keys</li>
</ul>
</div>
<div id="overview" class="section level2">
<h2>Overview</h2>
<p>Database indices improve the performance of queries against a database by organizing the data in a way that increase query execution.</p>
<p>This article was written to provide guidance on the setting of indices, primary and foreign keys for data that has been transformed into the Observational Medical Outcome Partnership (OMOP) Common Data Model (CDM). The community that supports the design and development of the OHDSI/CommonDataModel Github repository is a diverse collaborative of healthcare and technical profesisonals whom have limited data base adminstrative (DBA) experience. As a result, the comments below should be interpreted as suggestions and recommendations to help increase performance. Your teams needs may call for a modified configuration.</p>
</div>
<div id="general-recommendations" class="section level2">
<h2>General Recommendations</h2>
<p>Should your database of choice support indexing, the OMOP CDM Working Group recommends</p>
<ul>
<li>Indexing on all columns containing an "_id" (e.g. condition_occurrence_id, drug_exposure_id, measurement_id, procedure_occurrence_id, etc.)</li>
<li>Indexing on primary and foreign keys</li>
</ul>
<p>For all databases, regardless of custom indice support, primary and foreign keys should be set. This is a step towards ensuring data integrity.</p>
</div>
<div id="database-support" class="section level2">
<h2>Database support</h2>
<p>The OHDSI/CommonDataModel package leverages OHDSI/SQLRender and as a result is only capable of supporting sources that are supported by OHDSI/SQLRender. The following databases are currently supported.</p>
<div id="microsoft-sql-server" class="section level3">
<h3>Microsoft SQL Server</h3>
</div>
<div id="oracle" class="section level3">
<h3>Oracle</h3>
</div>
<div id="postgresql" class="section level3">
<h3>PostgreSQL</h3>
</div>
<div id="amazon-redshift" class="section level3">
<h3>Amazon Redshift</h3>
<p>On AWS Redshift it is important to ensure that your data is properly distributed and sorted across nodes. Compression on certain columns may also help. The designed DDL does set DISTKEYS in an effort to optimize performance. This configuration can be seen within the <a href="https://github.com/OHDSI/CommonDataModel/blob/v5.4/ddl/5.4/redshift/OMOPCDM_redshift_5.4_ddl.sql">Redshift-specific DDL</a>.</p>
</div>
<div id="impala" class="section level3">
<h3>Impala</h3>
</div>
<div id="ibm-netezza" class="section level3">
<h3>IBM Netezza</h3>
</div>
<div id="google-bigquery" class="section level3">
<h3>Google BigQuery</h3>
<p>Google BigQuery does not require manual optimization and/or sizing. Google BigQuery does massive parallel full table scans and intensive caching, all under the hood. <a href="https://forums.ohdsi.org/t/iso-best-practices-of-cdm-indexing/10939/2">Reference</a></p>
</div>
<div id="microsoft-parallel-data-warehouse-pdw" class="section level3">
<h3>Microsoft Parallel Data Warehouse (PDW)</h3>
</div>
<div id="sqlite" class="section level3">
<h3>SQLite</h3>
</div>
<div id="databricks" class="section level3">
<h3>Databricks</h3>
<p>This database type is not yet supported but is actively being worked on by a number of collaborators. For more informtion, please contact Ajit Londhe of Amgen.</p>
</div>
</div>
<div id="references" class="section level2">
<h2>References</h2>
<p><a href="https://forums.ohdsi.org/t/iso-best-practices-of-cdm-indexing/10939/2">ISO Best Practices of CDM Indexing</a></p>
</div>
</div>
</div>
</div>
<script>
// add bootstrap table styles to pandoc tables
function bootstrapStylePandocTables() {
$('tr.odd').parent('tbody').parent('table').addClass('table table-condensed');
}
$(document).ready(function () {
bootstrapStylePandocTables();
});
</script>
<!-- tabsets -->
<script>
$(document).ready(function () {
window.buildTabsets("TOC");
});
$(document).ready(function () {
$('.tabset-dropdown > .nav-tabs > li').click(function () {
$(this).parent().toggleClass('nav-tabs-open');
});
});
</script>
<!-- code folding -->
<script>
$(document).ready(function () {
// temporarily add toc-ignore selector to headers for the consistency with Pandoc
$('.unlisted.unnumbered').addClass('toc-ignore')
// move toc-ignore selectors from section div to header
$('div.section.toc-ignore')
.removeClass('toc-ignore')
.children('h1,h2,h3,h4,h5').addClass('toc-ignore');
// establish options
var options = {
selectors: "h1,h2,h3,h4,h5",
theme: "bootstrap3",
context: '.toc-content',
hashGenerator: function (text) {
return text.replace(/[.\\/?&!#<>]/g, '').replace(/\s/g, '_');
},
ignoreSelector: ".toc-ignore",
scrollTo: 0
};
options.showAndHide = true;
options.smoothScroll = true;
// tocify
var toc = $("#TOC").tocify(options).data("toc-tocify");
});
</script>
<!-- dynamically load mathjax for compatibility with self-contained -->
<script>
(function () {
var script = document.createElement("script");
script.type = "text/javascript";
script.src = "https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML";
document.getElementsByTagName("head")[0].appendChild(script);
})();
</script>
</body>
</html>

View File

@ -23,7 +23,6 @@ window.initializeCodeFolding = function(show) {
// create a collapsable div to wrap the code in // create a collapsable div to wrap the code in
var div = $('<div class="collapse r-code-collapse"></div>'); var div = $('<div class="collapse r-code-collapse"></div>');
var showThis = (show || $(this).hasClass('fold-show')) && !$(this).hasClass('fold-hide'); var showThis = (show || $(this).hasClass('fold-show')) && !$(this).hasClass('fold-hide');
if (showThis) div.addClass('in');
var id = 'rcode-643E0F36' + currentIndex++; var id = 'rcode-643E0F36' + currentIndex++;
div.attr('id', id); div.attr('id', id);
$(this).before(div); $(this).before(div);
@ -31,7 +30,7 @@ window.initializeCodeFolding = function(show) {
// add a show code button right above // add a show code button right above
var showCodeText = $('<span>' + (showThis ? 'Hide' : 'Code') + '</span>'); var showCodeText = $('<span>' + (showThis ? 'Hide' : 'Code') + '</span>');
var showCodeButton = $('<button type="button" class="btn btn-default btn-xs code-folding-btn pull-right"></button>'); var showCodeButton = $('<button type="button" class="btn btn-default btn-xs btn-secondary btn-sm code-folding-btn pull-right float-right"></button>');
showCodeButton.append(showCodeText); showCodeButton.append(showCodeText);
showCodeButton showCodeButton
.attr('data-toggle', 'collapse') .attr('data-toggle', 'collapse')
@ -47,13 +46,27 @@ window.initializeCodeFolding = function(show) {
div.before(buttonRow); div.before(buttonRow);
// show the div if necessary
if (showThis) div.collapse('show');
// update state of button on show/hide // update state of button on show/hide
div.on('hidden.bs.collapse', function () { // * Change text
// * add a class for intermediate states styling
div.on('hide.bs.collapse', function () {
showCodeText.text('Code'); showCodeText.text('Code');
showCodeButton.addClass('btn-collapsing');
});
div.on('hidden.bs.collapse', function () {
showCodeButton.removeClass('btn-collapsing');
}); });
div.on('show.bs.collapse', function () { div.on('show.bs.collapse', function () {
showCodeText.text('Hide'); showCodeText.text('Hide');
showCodeButton.addClass('btn-expanding');
}); });
div.on('shown.bs.collapse', function () {
showCodeButton.removeClass('btn-expanding');
});
}); });
} }

View File

@ -176,7 +176,7 @@
self._setEventHandlers(); self._setEventHandlers();
// Binding to the Window load event to make sure the correct scrollTop is calculated // Binding to the Window load event to make sure the correct scrollTop is calculated
$(window).load(function() { $(window).on("load", function() {
// Sets the active TOC item // Sets the active TOC item
self._setActiveElement(true); self._setActiveElement(true);

View File

@ -185,7 +185,7 @@ OBSERVATION,observation_source_value,No,varchar(50),"This field houses the verba
OBSERVATION,observation_source_concept_id,No,integer,"This is the concept representing the OBSERVATION_SOURCE_VALUE and may not necessarily be standard. This field is discouraged from use in analysis because it is not required to contain Standard Concepts that are used across the OHDSI community, and should only be used when Standard Concepts do not adequately represent the source detail for the Observation necessary for a given analytic use case. Consider using OBSERVATION_CONCEPT_ID instead to enable standardized analytics that can be consistent across the network.",If the OBSERVATION_SOURCE_VALUE is coded in the source data using an OMOP supported vocabulary put the concept id representing the source value here.,No,Yes,CONCEPT,CONCEPT_ID,,, OBSERVATION,observation_source_concept_id,No,integer,"This is the concept representing the OBSERVATION_SOURCE_VALUE and may not necessarily be standard. This field is discouraged from use in analysis because it is not required to contain Standard Concepts that are used across the OHDSI community, and should only be used when Standard Concepts do not adequately represent the source detail for the Observation necessary for a given analytic use case. Consider using OBSERVATION_CONCEPT_ID instead to enable standardized analytics that can be consistent across the network.",If the OBSERVATION_SOURCE_VALUE is coded in the source data using an OMOP supported vocabulary put the concept id representing the source value here.,No,Yes,CONCEPT,CONCEPT_ID,,,
OBSERVATION,unit_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the unit of the Observation that occurred. ,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,,,,, OBSERVATION,unit_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the unit of the Observation that occurred. ,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,,,,,
OBSERVATION,qualifier_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the qualifier of the Observation that occurred. ,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,,,,, OBSERVATION,qualifier_source_value,No,varchar(50),This field houses the verbatim value from the source data representing the qualifier of the Observation that occurred. ,This code is mapped to a Standard Condition Concept in the Standardized Vocabularies and the original code is stored here for reference.,No,No,,,,,
OBSERVATION,value_source_value,No,varchar(50),This field houses the verbatim result value of the Observation from the source data. Do not get confused with the Observation_source_value which captures source value of the observation mapped to observation_concept_id. This field is the observation result value from the source.,"If the observation_source_value was a question, for example, or an observation that requires a result then this field is the answer/ result from the source data. Store the verbatim value that represents the result of the observation_source_value.",No,No,,,,, OBSERVATION,value_source_value,No,varchar(50),This field houses the verbatim result value of the Observation from the source data. Do not get confused with the Observation_source_value which captures source value of the observation mapped to observation_concept_id. This field is the observation result value from the source.," If the observation_source_value was a question, for example, or an observation that requires a result then this field is the answer/ result from the source data. Store the verbatim value that represents the result of the observation_source_value. ",No,No,,,,,
DEATH,person_id,Yes,integer,,,No,Yes,PERSON,PERSON_ID,,, DEATH,person_id,Yes,integer,,,No,Yes,PERSON,PERSON_ID,,,
DEATH,death_date,Yes,date,The date the person was deceased.,"If the precise date include day or month is not known or not allowed, December is used as the default month, and the last day of the month the default day.",No,No,,,,, DEATH,death_date,Yes,date,The date the person was deceased.,"If the precise date include day or month is not known or not allowed, December is used as the default month, and the last day of the month the default day.",No,No,,,,,
DEATH,death_datetime,No,datetime,,If not available set time to midnight (00:00:00),No,No,,,,, DEATH,death_datetime,No,datetime,,If not available set time to midnight (00:00:00),No,No,,,,,
@ -361,11 +361,13 @@ Condition.",,No,No,,,,,
CONDITION_ERA,condition_occurrence_count,No,integer,"The number of individual Condition CONDITION_ERA,condition_occurrence_count,No,integer,"The number of individual Condition
Occurrences used to construct the Occurrences used to construct the
condition era.",,No,No,,,,, condition era.",,No,No,,,,,
METADATA,metadata_id,Yes,integer,The unique key given to a Metadata record.,Attribute value is auto-generated,Yes,No,,,,,
METADATA,metadata_concept_id,Yes,integer,,,No,Yes,CONCEPT,CONCEPT_ID,,, METADATA,metadata_concept_id,Yes,integer,,,No,Yes,CONCEPT,CONCEPT_ID,,,
METADATA,metadata_type_concept_id,Yes,integer,,,No,Yes,CONCEPT,CONCEPT_ID,,, METADATA,metadata_type_concept_id,Yes,integer,,,No,Yes,CONCEPT,CONCEPT_ID,,,
METADATA,name,Yes,varchar(250),,,No,No,,,,, METADATA,name,Yes,varchar(250),,,No,No,,,,,
METADATA,value_as_string,No,varchar(250),,,No,No,,,,, METADATA,value_as_string,No,varchar(250),,,No,No,,,,,
METADATA,value_as_concept_id,No,integer,,,No,Yes,CONCEPT,CONCEPT_ID,,, METADATA,value_as_concept_id,No,integer,,,No,Yes,CONCEPT,CONCEPT_ID,,,
METADATA,value_as_number,No,float,"This is the numerical value of the Result of the Metadata, if applicable and available. It is not expected that all Metadata will have numeric results, rather, this field is here to house values should they exist. ",,No,No,,,,,
METADATA,metadata_date,No,date,,,No,No,,,,, METADATA,metadata_date,No,date,,,No,No,,,,,
METADATA,metadata_datetime,No,datetime,,,No,No,,,,, METADATA,metadata_datetime,No,datetime,,,No,No,,,,,
CDM_SOURCE,cdm_source_name,Yes,varchar(255),The name of the CDM instance.,,No,No,,,,, CDM_SOURCE,cdm_source_name,Yes,varchar(255),The name of the CDM instance.,,No,No,,,,,

1 cdmTableName cdmFieldName isRequired cdmDatatype userGuidance etlConventions isPrimaryKey isForeignKey fkTableName fkFieldName fkDomain fkClass unique DQ identifiers
185 NOTE note_datetime No datetime If time is not given set the time to midnight. No No
186 NOTE note_type_concept_id Yes integer The provenance of the note. Most likely this will be EHR. Put the source system of the note, as in EHR record. [Accepted Concepts](https://athena.ohdsi.org/search-terms/terms?standardConcept=Standard&domain=Type+Concept&page=1&pageSize=15&query=). No Yes CONCEPT CONCEPT_ID Type Concept
187 NOTE note_class_concept_id Yes integer A Standard Concept Id representing the HL7 LOINC Document Type Vocabulary classification of the note. Map the note classification to a Standard Concept. For more information see the ETL Conventions in the description of the NOTE table. [Accepted Concepts](https://athena.ohdsi.org/search-terms/terms?standardConcept=Standard&conceptClass=Doc+Kind&conceptClass=Doc+Role&conceptClass=Doc+Setting&conceptClass=Doc+Subject+Matter&conceptClass=Doc+Type+of+Service&domain=Meas+Value&page=1&pageSize=15&query=). This Concept can alternatively be represented by concepts with the relationship 'Kind of (LOINC)' to [706391](https://athena.ohdsi.org/search-terms/terms/706391) (Note). No Yes CONCEPT CONCEPT_ID
188 NOTE note_title No varchar(250) The title of the note. No No
189 NOTE note_text Yes varchar(MAX) The content of the note. No No
190 NOTE encoding_concept_id Yes integer This is the Concept representing the character encoding type. Put the Concept Id that represents the encoding character type here. Currently the only option is UTF-8 ([32678](https://athena.ohdsi.org/search-terms/terms/32678)). It the note is encoded in any other type, like ASCII then put 0. No Yes CONCEPT CONCEPT_ID
191 NOTE language_concept_id Yes integer The language of the note. Use Concepts that are descendants of the concept [4182347](https://athena.ohdsi.org/search-terms/terms/4182347) (World Languages). No Yes CONCEPT CONCEPT_ID
361 CONCEPT_RELATIONSHIP relationship_id concept_id_1 Yes varchar(20) integer The relationship between CONCEPT_ID_1 and CONCEPT_ID_2. Please see the [Vocabulary Conventions](https://ohdsi.github.io/CommonDataModel/dataModelConventions.html#concept_relationships). for more information. No Yes RELATIONSHIP CONCEPT RELATIONSHIP_ID CONCEPT_ID
362 CONCEPT_RELATIONSHIP valid_start_date concept_id_2 Yes date integer The date when the relationship is first recorded. No No Yes CONCEPT CONCEPT_ID
363 CONCEPT_RELATIONSHIP valid_end_date relationship_id Yes date varchar(20) The date when the relationship is invalidated. The relationship between CONCEPT_ID_1 and CONCEPT_ID_2. Please see the [Vocabulary Conventions](https://ohdsi.github.io/CommonDataModel/dataModelConventions.html#concept_relationships). for more information. No No Yes RELATIONSHIP RELATIONSHIP_ID
364 CONCEPT_RELATIONSHIP valid_start_date Yes date The date when the relationship is first recorded. No No
365 CONCEPT_RELATIONSHIP invalid_reason valid_end_date No Yes varchar(1) date Reason the relationship was invalidated. Possible values are 'D' (deleted), 'U' (updated) or NULL. The date when the relationship is invalidated. No No
366 RELATIONSHIP CONCEPT_RELATIONSHIP relationship_id invalid_reason Yes No varchar(20) varchar(1) The type of relationship captured by the relationship record. Reason the relationship was invalidated. Possible values are 'D' (deleted), 'U' (updated) or NULL. Yes No No
367 RELATIONSHIP relationship_name relationship_id Yes varchar(255) varchar(20) The type of relationship captured by the relationship record. No Yes No
368 RELATIONSHIP is_hierarchical relationship_name Yes varchar(1) varchar(255) Defines whether a relationship defines concepts into classes or hierarchies. Values are 1 for hierarchical relationship or 0 if not. No No
369 RELATIONSHIP defines_ancestry is_hierarchical Yes varchar(1) Defines whether a hierarchical relationship contributes to the concept_ancestor table. These are subsets of the hierarchical relationships. Valid values are 1 or 0. Defines whether a relationship defines concepts into classes or hierarchies. Values are 1 for hierarchical relationship or 0 if not. No No
370 RELATIONSHIP defines_ancestry Yes varchar(1) Defines whether a hierarchical relationship contributes to the concept_ancestor table. These are subsets of the hierarchical relationships. Valid values are 1 or 0. No No
371 RELATIONSHIP reverse_relationship_id Yes varchar(20) The identifier for the relationship used to define the reverse relationship between two concepts. No No
372 RELATIONSHIP relationship_concept_id Yes integer A foreign key that refers to an identifier in the [CONCEPT](https://ohdsi.github.io/CommonDataModel/cdm531.html#concept) table for the unique relationship concept. No Yes CONCEPT CONCEPT_ID
373 CONCEPT_SYNONYM concept_id Yes integer No Yes CONCEPT CONCEPT_ID

View File

@ -27,10 +27,10 @@ navbar:
menu: menu:
- text: "CDM v3.0" - text: "CDM v3.0"
href: cdm30.html href: cdm30.html
- text: "CDM v5.3.1" - text: "CDM v5.3"
href: cdm531.html href: cdm53.html
- text: "CDM v5.4" - text: "CDM v5.4"
href: cdm531.html href: cdm54.html
- text: "CDM v6.0" - text: "CDM v6.0"
href: cdm60.html href: cdm60.html
- text: "Proposals" - text: "Proposals"

109
rmd/cdm53.Rmd Normal file
View File

@ -0,0 +1,109 @@
---
output:
# pdf_document:
# toc: yes
# toc_depth: '5'
html_document:
toc: yes
toc_depth: 5
toc_float: yes
---
```{r setup, include=FALSE, eval=TRUE}
#todo figure out how to get the document dynamically without specifying the name
library(rmarkdown)
library(knitr)
library(kableExtra)
library(magrittr)
library(dplyr)
library(stringr)
```
# **OMOP CDM v5.3**
Below is the specification document for the OMOP Common Data Model, v5.3 (previously v5.3.1). Each table is represented with a high-level description and ETL conventions that should be followed. This is continued with a discussion of each field in each table, any conventions related to the field, and constraints that should be followed (like primary key, foreign key, etc). Should you have questions please feel free to visit the [forums](https://forums.ohdsi.org/) or the [github issue](https://github.com/ohdsi/CommonDataModel/issues) page.
*__Special Note__ This documentation previously referenced v5.3.1. During the OHDSI/CommonDataModel Hack-A-Thon that occurred on August 18, 2021 the decision was made to align documentation with the minor releases. Hot fixes and minor.minor release can be found through the searching of tags.*
--after regeneration of DDLs
link to csv of cdm
link to pdf of cdm documentation
link to forum on doc page
```{r docLoop53, echo=FALSE, results='asis'}
tableSpecs <- read.csv("../inst/csv/OMOP_CDMv5.3_Table_Level.csv", stringsAsFactors = FALSE)
cdmSpecs <- read.csv("../inst/csv/OMOP_CDMv5.3_Field_Level.csv", stringsAsFactors = FALSE)
tables <- tableSpecs$cdmTableName
cdmSpecsClean <- cdmSpecs %>%
dplyr::select(`CDM Table` = cdmTableName,
`CDM Field` = cdmFieldName,
`User Guide` = userGuidance,
`ETL Conventions` = etlConventions,
`Datatype` = cdmDatatype,
`Required` = isRequired,
`Primary Key` = isPrimaryKey,
`Foreign Key` = isForeignKey,
`FK Table` = fkTableName,
`FK Domain` = fkDomain
)
cdmSpecsClean[is.na(cdmSpecsClean)] <- ""
for(tb in tables) {
if(tb == 'PERSON'){
cat("## **Clinical Data Tables**\n\n")
}
if(tb == 'LOCATION'){
cat("## **Health System Data Tables**\n\n")
}
if(tb == 'PAYER_PLAN_PERIOD'){
cat("## **Health Economics Data Tables**\n\n")
}
if(tb == 'DRUG_ERA'){
cat("## **Standardized Derived Elements**\n\n")
}
if(tb == 'METADATA'){
cat("## **Metadata Tables**\n\n")
}
if(tb == 'CONCEPT'){
cat("## **Vocabulary Tables**\n\n")
}
cat("###", tb, "{.tabset .tabset-pills} \n\n")
tableInfo <- subset(tableSpecs, cdmTableName == tb)
cat("**Table Description**\n\n",tableInfo[,"tableDescription"], "\n\n")
if(!isTRUE(tableInfo[,"userGuidance"]=="")){
cat("**User Guide**\n\n",tableInfo[,"userGuidance"],"\n\n")
}
if(!isTRUE(tableInfo[,"etlConventions"]=="")){
cat("**ETL Conventions**\n\n",tableInfo[,"etlConventions"],"\n\n")
}
loopTable <- subset(cdmSpecsClean, `CDM Table` == tb)
loopTable <- subset(loopTable, select = -c(1))
print(kable(x = loopTable, align = "l", row.names = FALSE, format = "html", escape = FALSE) %>%
column_spec(1, bold = T) %>%
column_spec(2, width = "3in", include_thead = T) %>%
column_spec(3, width = "4in", include_thead = T) %>%
column_spec(4:9, width = "1in", include_thead = T) %>%
kable_styling(c("condensed","hover"), position = "center", full_width = T, font_size = 13))
}
```

View File

@ -21,9 +21,9 @@ library(stringr)
``` ```
# **OMOP CDM v5.3.1** # **OMOP CDM v5.4**
Below is the specification document for the OMOP Common Data Model, v5.3.1. Each table is represented with a high-level description and ETL conventions that should be followed. This is continued with a discussion of each field in each table, any conventions related to the field, and constraints that should be followed (like primary key, foreign key, etc). Should you have questions please feel free to visit the [forums](https://forums.ohdsi.org/) or the [github issue](https://github.com/ohdsi/CommonDataModel/issues) page. Below is the specification document for the OMOP Common Data Model, v5.4. Each table is represented with a high-level description and ETL conventions that should be followed. This is continued with a discussion of each field in each table, any conventions related to the field, and constraints that should be followed (like primary key, foreign key, etc). Should you have questions please feel free to visit the [forums](https://forums.ohdsi.org/) or the [github issue](https://github.com/ohdsi/CommonDataModel/issues) page.
--after regeneration of DDLs --after regeneration of DDLs
link to csv of cdm link to csv of cdm
@ -31,8 +31,8 @@ link to pdf of cdm documentation
link to forum on doc page link to forum on doc page
```{r docLoop53, echo=FALSE, results='asis'} ```{r docLoop53, echo=FALSE, results='asis'}
tableSpecs <- read.csv("../inst/csv/OMOP_CDMv5.3.1_Table_Level.csv", stringsAsFactors = FALSE) tableSpecs <- read.csv("../inst/csv/OMOP_CDMv5.4_Table_Level.csv", stringsAsFactors = FALSE)
cdmSpecs <- read.csv("../inst/csv/OMOP_CDMv5.3.1_Field_Level.csv", stringsAsFactors = FALSE) cdmSpecs <- read.csv("../inst/csv/OMOP_CDMv5.4_Field_Level.csv", stringsAsFactors = FALSE)
tables <- tableSpecs$cdmTableName tables <- tableSpecs$cdmTableName

View File

@ -0,0 +1,48 @@
---
title: "Indices, Primary Keys and Foreign Key Constraints"
output:
html_document:
toc: true
toc_depth: 5
toc_float: true
---
## Overview
Database indices improve the performance of queries against a database by organizing the data in a way that increase query execution.
This article was written to provide guidance on the setting of indices, primary and foreign keys for data that has been transformed into the Observational Medical Outcome Partnership (OMOP) Common Data Model (CDM). The community that supports the design and development of the OHDSI/CommonDataModel Github repository is a diverse collaborative of healthcare and technical profesisonals whom have limited data base adminstrative (DBA) experience. As a result, the comments below should be interpreted as suggestions and recommendations to help increase performance. Your teams needs may call for a modified configuration.
## General Recommendations
Should your database of choice support indexing, the OMOP CDM Working Group recommends
* Indexing on all columns containing an "_id" (e.g. condition_occurrence_id, drug_exposure_id, measurement_id, procedure_occurrence_id, etc.)
* Indexing on primary and foreign keys
For all databases, regardless of custom indice support, primary and foreign keys should be set. This is a step towards ensuring data integrity. Information on what table level attributes should be set as primary and foreign keys can be found within the *_Field_Level.csv file(s) located in the [INST/CSV directory](https://github.com/OHDSI/CommonDataModel/tree/v5.4/inst/csv)
## Database support
The OHDSI/CommonDataModel package leverages OHDSI/SQLRender and as a result is only capable of supporting sources that are supported by OHDSI/SQLRender. The following databases are currently supported.
### Microsoft SQL Server
### Oracle
### PostgreSQL
### Amazon Redshift
On AWS Redshift it is important to ensure that your data is properly distributed and sorted across nodes. Compression on certain columns may also help. The designed DDL does set DISTKEYS in an effort to optimize performance. This configuration can be seen within the [Redshift-specific DDL](https://github.com/OHDSI/CommonDataModel/blob/v5.4/ddl/5.4/redshift/OMOPCDM_redshift_5.4_ddl.sql).
### Impala
### IBM Netezza
### Google BigQuery
Google BigQuery does not require manual optimization and/or sizing. Google BigQuery does massive parallel full table scans and intensive caching, all under the hood.
[Reference](https://forums.ohdsi.org/t/iso-best-practices-of-cdm-indexing/10939/2)
### Microsoft Parallel Data Warehouse (PDW)
### SQLite
### Databricks
This database type is not yet supported but is actively being worked on by a number of collaborators. For more informtion, please contact Ajit Londhe of Amgen.
## References
[ISO Best Practices of CDM Indexing](https://forums.ohdsi.org/t/iso-best-practices-of-cdm-indexing/10939/2)