data-transport/README.md

24 lines
1.2 KiB
Markdown
Raw Normal View History

2017-08-07 15:21:00 +00:00
# Introduction
2017-08-07 15:06:12 +00:00
2024-04-18 04:56:31 +00:00
This project implements an abstraction of objects that can have access to a variety of data stores, implementing read/write with a simple and expressive interface. This abstraction works with **NoSQL**, **SQL** and **Cloud** data stores and leverages **pandas**.
2022-01-29 23:01:43 +00:00
# Why Use Data-Transport ?
2024-04-24 18:00:03 +00:00
Mostly data scientists that don't really care about the underlying database and would like a simple and consistent way to read/write and move data are well served. Additionally we implemented lightweight Extract Transform Loading API and command line (CLI) tool. Finally it is possible to add pre/post processing pipeline functions to read/write
2022-01-29 23:01:43 +00:00
1. Familiarity with **pandas data-frames**
2. Connectivity **drivers** are included
2024-06-14 19:16:06 +00:00
3. Reading/Writing data from various sources
2024-04-24 18:00:03 +00:00
4. Useful for data migrations or **ETL**
2017-08-07 15:21:00 +00:00
2022-01-29 23:18:20 +00:00
## Installation
2019-09-17 16:53:44 +00:00
2022-01-29 23:18:20 +00:00
Within the virtual environment perform the following :
2019-09-17 16:53:44 +00:00
2022-12-14 22:48:40 +00:00
pip install git+https://github.com/lnyemba/data-transport.git
2019-09-17 16:53:44 +00:00
2017-08-07 15:21:00 +00:00
2024-04-18 04:56:31 +00:00
## Learn More
2022-01-29 23:01:43 +00:00
2024-04-18 04:56:31 +00:00
We have available notebooks with sample code to read/write against mongodb, couchdb, Netezza, PostgreSQL, Google Bigquery, Databricks, Microsoft SQL Server, MySQL ... Visit [data-transport homepage](https://healthcareio.the-phi.com/data-transport)