hiplab/data-transport
hiplab
/
data-transport
mirror of https://github.com/lnyemba/data-transport
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
240 Commits 7 Branches 0 Tags 577 KiB
Commit Graph

178 Commits

Author SHA1 Message Date
Steve Nyemba 6b78a82e99 bug fix 2022-06-10 10:27:44 -05:00
Steve Nyemba b2a224f4a7 bug fix: sql lock for parallel processing 2022-06-10 10:11:37 -05:00
Steve Nyemba 40bcfc40f9 bug fix: authentication 2022-05-16 13:14:29 -05:00
Steve Nyemba 7c328e5106 bug fix: mongodb mechanism parameter for authentication 2022-05-16 11:59:46 -05:00
Steve Nyemba db1496dd38 bug fixes 2022-05-16 11:54:28 -05:00
Steve Nyemba f64b945245 new: authentication using file on disk, misc bug fixes 2022-05-16 11:27:36 -05:00
Steve Nyemba 8cd34d902a bug fix: ETL logging and rabbitmq-server listener 2022-05-11 11:17:27 -05:00
Steve Nyemba 67cb7de861 bug fix: logging with ETL 2022-05-06 14:30:59 -05:00
Steve Nyemba 899339f11f bug fix: authentication mongodb 2022-05-06 14:25:12 -05:00
Steve Nyemba fd9523e99a enhancement mongodb 2022-04-29 11:15:32 -05:00
Steve Nyemba e9a6779a9b new file and documentation 2022-04-11 23:31:47 -05:00
Steve Nyemba 98eaa99820 bug fixes and enhancements, added console output(for logging) 2022-04-11 18:34:32 -05:00
Steve Nyemba 6c406407b2 bug fix: ... minor update sqlalchemy 2022-03-31 17:13:24 -05:00
Steve Nyemba e5fadc64a0 optimizations mongodb 2022-03-19 00:02:53 -05:00
Steve Nyemba 38e1bce6c2 bug fixes and optimizations 2022-03-12 12:25:29 -06:00
Steve Nyemba 105ff00224 bugfix: ETL multiprocessing 2022-03-07 18:50:29 -06:00
Steve Nyemba f672c04844 bugfix: sqlalchemy 2022-03-07 15:29:03 -06:00
Steve Nyemba cfc683c1b3 bug fix: transport cli and write function for sql 2022-03-07 14:17:27 -06:00
Steve Nyemba fd9442c298 bug fix: write problems, using drivers 2022-03-04 15:00:30 -06:00
Steve Nyemba 14a551e57b Bug fix: sqlalchemy facilities added 2022-03-03 16:08:24 -06:00
Steve Nyemba a7df7bfbce bug fix: has sql 2022-03-02 13:15:35 -06:00
Steve Nyemba b239a5149f bug fixes and simplifying interfaces 2022-01-29 11:15:45 -06:00
Steve Nyemba ccc05acc01 bug fix: etl engine, sqlite inserts 2021-12-09 15:25:58 -06:00
Steve Nyemba 2de6e51bdb bug fix: simplyfying factory interface 2021-11-18 15:21:26 -06:00
Steve Nyemba 44ef71eb92 bug fix 2021-09-03 01:25:45 -05:00
Steve Nyemba 0c0eaf7063 bug fix: imports, testing code removed 2021-09-01 02:33:40 -05:00
Steve Nyemba 83cc1778c7 bug fix: print 2021-07-28 22:51:19 -05:00
Steve Nyemba 10adde7a08 bug fix: sqlwriter 2021-07-28 22:32:07 -05:00
Steve Nyemba 79cdc0c0d0 bug fix, enhancement with pandas 2021-07-23 15:22:23 -05:00
Steve Nyemba 835e1253a5 bug fix: filereader 2021-07-19 19:07:06 -05:00
Steve Nyemba b21ddd6945 bug fix: removing testing code 2021-07-19 18:54:37 -05:00
Steve Nyemba 19ab510125 cli etl tool 2021-07-08 17:31:29 -05:00
Steve Nyemba 7c2e945996 bug fix: support for netezza 2021-06-29 15:12:28 -05:00
Steve Nyemba 4c810fe57a bug fix 2021-06-20 18:56:12 -05:00
Steve Nyemba b5eb5e0991 bug fix: adding types to read function 2021-06-15 12:30:48 -05:00
Steve Nyemba 91434ec32d bug fix: pandas dtypes 2021-06-14 14:31:30 -05:00
Steve Nyemba cebc784494 bug fix: lock, parallel processing bigquery 2021-04-13 17:29:54 -05:00
Steve Nyemba 84e212d5a3 bug fix: lock, parallel processing bigquery 2021-04-13 17:27:23 -05:00
Steve Nyemba afcc5ed690 bug fix ... 2021-03-29 23:30:54 -05:00
Steve Nyemba 687ffec215 bq support 2021-03-15 12:38:54 -05:00
Steve Nyemba 5bf2012c59 bug fix: batch update fixes 2021-01-10 09:55:04 -06:00
Steve Nyemba fa615aa931 bug fix: insert on an etl with data-typing (tricky) 2021-01-02 17:39:27 -06:00
Steve Nyemba eef84d8069 bug fix: sql writer::apply 2021-01-02 05:38:06 -06:00
Steve Nyemba 898455f23c bug fix: sql writer::apply 2021-01-02 05:35:49 -06:00
Steve Nyemba 18c9245ace bug fix: sql writer 2021-01-02 05:29:52 -06:00
Steve Nyemba 99d29f0a1f bug fix: sql writer 2021-01-02 05:24:12 -06:00
Steve Nyemba 504596f838 new: added sql handler for Redshift,PostgreSQL, MySQL/Mariadb 2020-12-31 11:12:34 -06:00
Steve Nyemba 7ba09309d3 bug fix: mongodb commands with values 2020-12-26 12:20:40 -06:00
Steve Nyemba 03e0203c28 bug fix: sqlite lock 2020-12-11 03:22:49 -06:00
Steve Nyemba 6f08d64e49 bug fix: sqlite import pandas 2020-12-11 02:46:31 -06:00
Steve Nyemba 2be9d9db2f bug fix: aws s3 handler 2020-12-10 03:56:08 -06:00
Steve Nyemba c36b07073a bug fix with s3 hierarchy 2020-12-10 03:42:30 -06:00
Steve Nyemba 4b49b0a060 bug fix with mongodb client 2020-11-15 21:19:40 -06:00
Steve Nyemba 9eb46954cd bug fixes: mongodb and s3 bucket 2020-11-04 14:26:46 -06:00
Steve Nyemba e70802ef00 mongo thread/process safe? 2020-10-08 17:14:35 -05:00
Steve Nyemba 18d6ff90fe close function for mongodb 2020-10-06 15:26:06 -05:00
Steve Nyemba e4a1ef8dd7 bug fix 2020-09-26 16:53:33 -05:00
Steve Nyemba cbf641a986 bug fix with reader 2020-09-22 13:30:04 -05:00
Steve Nyemba 5676c4991f json object writing ... 2020-08-11 17:20:46 -05:00
Steve Nyemba 4a985e82dc bug fix with diskwriter (adding return carriage) 2020-08-11 16:42:21 -05:00
Steve Nyemba 617f829e1e bug fix:sqlitewriter 2020-07-27 21:47:02 -05:00
Steve Nyemba 21ef842137 bug fix 2020-05-17 22:53:54 -05:00
Steve Nyemba d5ba648abf support for filters in read 2020-05-17 21:57:18 -05:00
Steve Nyemba 834dc80e79 bug fix (updating pika interface) 2020-03-04 02:23:21 -06:00
Steve Nyemba 1bb7c1bd0a Bug fix with queueReader 2020-03-02 11:33:35 -06:00
Steve Nyemba 172b609b0e bug fix 2020-02-10 15:48:23 -06:00
Steve Nyemba a16b969b69 bug fix & aliasing 2020-02-05 18:14:30 -06:00
Steve Nyemba c268a117c2 bug fixes with upgraded version of pika 2020-02-05 15:06:35 -06:00
Steve Nyemba 081ed080d7 bug fixes for version 1.0.8, streamlining interface 2020-02-01 09:42:45 -06:00
Steve Nyemba aaad4003a9 bug fix 2019-12-01 21:32:47 -06:00
Steve Nyemba b0380be861 bug fix with file writer 2019-12-01 21:23:54 -06:00
Steve Nyemba 008ae6edf9 bug fix with mongodb 2019-11-05 16:05:06 -06:00
Steve Nyemba ac64f8de95 bug fixes 2019-11-05 16:04:54 -06:00
Steve Nyemba d0a0334f7a compatibility to python 3.6 ... 2019-11-04 21:51:20 -06:00
Steve Nyemba 0c6d6956aa housekeeping/documentation 2019-09-17 12:00:45 -05:00
Steve Nyemba e8328e577d bug fix 2019-09-17 11:21:42 -05:00
Steve Nyemba bc8f24d3cc data transport framework for rabbitmq, mongodb, couchdb, ... 2019-09-16 23:09:16 -05:00
Steve Nyemba 0b15351b8e data transport framework for rabbitmq, mongodb, couchdb, ... 2019-09-16 23:08:43 -05:00