parser/README.md

About Healthcare/IO Parser

The Healthcare/IO parser is an Electronic Data Interchange (EDI) parser developed at Vanderbilt University Medical Center during Khanhly Nguyen's summer internship 2019. Built in a healthcare setting, the parser focuses (for now) on x12 claims (837) and remittances (835)

This code is intended to extract x12 837 and 835 and format them into portable and human readable format (JSON). This allows the claims to be stored in document data stores such as Mongodb, couchdb or databases that have support for JSON like PostgreSQL

We wrote this frame to be used in both command line or as a library within in your code. The framework is driven by configurations that derviced from X12 standards.

Features

Features	Description
X12 claims/remits	parsing of {x12} claims/remittances into JSON format with human readible attributes
Multi Processing	capable of processing multiple files simultaneously to speed up processing
Analytics support	descriptive statistical analytics : distribution, various counts
Process Recovery	capable of recovering interrupted runs
Export to RDBMS	exports data to relational format (NoSQL -> SQL) 7 supported databases * PostgreSQL, * Redshift, * Neteeza, * Mysql, * Mariadb, * bigquery, * sqlite3
Issues and Bug reports	info@the-phi.com

For advanced features visit Healthcare/IO::Parser homepage

Installation

System requirements

- git (latest cli)
- python3+ (with virtualenv)
- mongodb (optional)

Installation command

pip install --upgrade git+https://hiplab.mc.vanderbilt.edu/git/lab/parse-edi.git

Usage

Healthcare/IO is primarily intended to be used as a command line parser (for now). It is fully written in python 3+

CLI :

1. signup to get parsing configuration

   The parser is driven by a configuration file that specifies fields to parse and how to parse them. You need by signing up, to get a copy of the configuration file. # # Use sqlite as data-store healthcare-io.py --signup [--store <mongo|sqlite>]

2. check version

   Occasionally the attributes in the configuration file may change, This function will determine if there is a new version available.

    healthcare-io.py --check-update
   

3. parsing data stored in a folder

   The parser will recursively traverse a directory with claims and or remittances

    healthcare-io.py --parse --folder <path> [--batch <n>] [--resume]
   
    with :
        --parse     tells the engine what to parse claims or remits
        --folder    location of the claims|remits
        --batch     number of processes to spawn to parse the files
        --resume    tells the parser to resume parsing 
                    if all files weren't processed or new files were added into the folder
   

4. export data to a relational data-store

   The parser will export data into other data-stores as a relational tables allowing users to construct views to support a variety of studies.

    healthcare-io.py --export <835|837> --export-config <path-export.json>
   
    with:
        --export-config    configuration to support data-store
   

   example 1. Exporting to PostgreSQL

        {"provider":"postgresql","database":"healthcareio","schema":"foo"}
   

   NOTE

   The output generates a set of tables that are the result of transforming unstructured data to relational structure. The tables can be bound with the attribute _id

   The configuration file needed to implement export is modelled after the following template:

    {
        "provider":"<postgresql|redshift|mysql|mariadb>",
        "database":"<name>",["host":"server-name"],["port":5432],
        ["user":"me"],["password":"!@#z4qm"],["schema":"target-schema"]
   
        }
   

   parameters:

    provider    postgresql,redshift,mysql or mariadb (supported providers)
    database    name of the database
   

   optional:

    schema      name of the target schema. If not provided we will assume the default
    host        host of the database. If not provided assuming localhost
    port        port value of the database if not provided the default will be used
    user        database user name. If not provided we assume security settings to trust
    password    password of database user. If not set we assume security settings to trust
   

Known Limitations

1. By default it does NOT come with all {X12} Segments.
2. Does not support an easy way to rename attributes it parses
3. Upgrade configuration may require dropping tables
4. For now can only read {x12} from disk

There is support for additional features and attributes available at Healthcare/IO::Parser.

In development

1. Wizard/UI to enable attribute renaming
2. Dashboard for quick overview
3. Reading {x12} from s3 and other cloud buckets
4. Docker Image

Credits

This project was realized thanks largely to the effort of Vanderbilt University Medical Center

- Khanhly Nguyen - Heather Dunn - Nina Thousand	- Byun Kang - Gaylon Stanley - Cheng Gao	- Thomas Brown - Brad Malin - Steve Nyemba