Commit Graph

188 Commits

Author SHA1 Message Date
Steve Nyemba 88b0437bbc bug fix: etl jobs 2023-12-26 14:51:28 -06:00
Steve Nyemba 81bc5a3ba1 bug fix: bigquery chunk/batch sizes 2023-12-22 14:02:32 -06:00
Steve Nyemba fbfaaebbdc adding alias CALLBACK == QLISTENER 2023-12-20 10:14:15 -06:00
Steve Nyemba e46ebadcc2 bug fix 2023-12-11 22:10:53 -06:00
Steve Nyemba d74372f645 bug fixes: mongodb, common, nextcloud 2023-12-08 18:19:46 -06:00
Steve Nyemba 56bdda17b7 minor bug fix 2023-11-30 12:40:57 -06:00
Steve Nyemba 5a5922b736 version update 2023-11-30 12:12:14 -06:00
Steve Nyemba 214f276ae4 bug fixes with sqlite and provider 2023-11-30 12:05:44 -06:00
Steve Nyemba 9da2894b07 bug fixes with sqlite and provider 2023-11-30 12:05:04 -06:00
Steve Nyemba 7d29a69a23 reminder 2023-11-21 11:01:05 -06:00
Steve Nyemba 5660d8ba59 nextcloud handling 2023-11-11 10:30:58 -06:00
Steve Nyemba a7fe357b2c bug fixes, write file and other misc with ETL 2023-09-30 09:24:58 -05:00
Steve Nyemba 4320159f3d bug fixes 2023-09-30 01:17:35 -05:00
Steve Nyemba 2bb07aedec bug fix: sqlite and cursors and transport 2023-09-30 00:18:37 -05:00
Steve Nyemba 3f7f3d7306 refactor: factory, etl, fixes: session 2023-09-29 20:27:53 -05:00
Steve Nyemba 324d81bd16 bug fix with mysql 2023-09-18 20:00:40 -05:00
Steve Nyemba 0da32069e2 databricks support 2023-09-08 11:28:35 -05:00
Steve Nyemba 7b25e392b0 bug fixes 2023-08-15 15:27:19 -05:00
Steve Nyemba 1ebe253595 bug fix 2023-08-14 11:46:18 -05:00
Steve Nyemba bcb65d873a bug fixes: console/cli handling 2023-08-14 11:46:18 -05:00
Steve Nyemba 6fedef41a3 version update 2023-07-28 09:10:18 -05:00
Steve Nyemba 3b5c9d1503 bugfix: reformatting factory singleton and adding python queue as qlistener 2023-07-28 09:09:18 -05:00
Steve Nyemba f4c0ca80aa bug fix version 2023-07-17 15:46:14 -05:00
Steve Nyemba fe21f44388 bug fix version 2023-07-17 15:42:42 -05:00
Steve Nyemba 05d990d713 bug fix: table creation with schema (if any is provided) 2023-07-09 21:22:31 -05:00
Steve Nyemba 8f94ea260b bug fix: initialization of fields unnecessary 2023-07-09 20:58:00 -05:00
Steve Nyemba 653d22cd8c bug fix: init function for fields not working 2023-07-09 20:47:54 -05:00
Steve Nyemba 0933e4155f bug fixes has function 2023-07-09 18:03:03 -05:00
Steve Nyemba b5794e0a61 bug fixes has function 2023-07-09 17:58:32 -05:00
Steve Nyemba ad84e24533 version update 2023-07-09 17:51:47 -05:00
Steve Nyemba 1885e3a5f6 bug fix, making table when provided a schema 2023-07-09 17:51:09 -05:00
Steve Nyemba f57cd8d871 version update 2023-07-09 17:46:01 -05:00
Steve Nyemba 5f1592b973 bug fix 2023-07-09 17:37:57 -05:00
Steve Nyemba fb20687a46 version update 2023-07-09 16:37:30 -05:00
Steve Nyemba 712c3d076d bug fix: schema specification in sql handling 2023-07-09 16:32:05 -05:00
Steve Nyemba 687b211b7b bug fix: field_name (typographic error) 2023-07-05 15:58:51 -05:00
Steve Nyemba b27c5a88d4 bug fix: bigquery meta on a project other than the one with service key 2023-07-05 15:53:54 -05:00
Steve Nyemba 74d8b2c6f2 bug fix 2023-07-05 14:51:14 -05:00
Steve Nyemba aaaf64db33 bug fix: versions of sqlalchemy for meta data 2023-06-16 17:39:02 -05:00
Steve Nyemba 18cd1d8b11 bug fix with sql handling sqlalchemy 2023-06-16 16:07:29 -05:00
Steve Nyemba 66f43a98c1 bug fixes: etl, mongodb lexicon 2023-05-30 15:47:10 -05:00
Steve Nyemba 78ad8f3ad8 bug fix: etl transport 2023-05-30 15:19:10 -05:00
Steve Nyemba ed66370fdf bug fix: ETL jobs and streamline 2023-05-30 15:01:51 -05:00
Steve Nyemba 695c10e797 bug fix, added ferretdb 2023-05-25 09:39:51 -05:00
Steve Nyemba 5f9416d2f6 bug fix: removing unused library 2023-01-18 21:13:38 -06:00
Steve Nyemba dcd6a83a35 bug fix: disk - nujson, refactor 2023-01-04 09:12:55 -06:00
Steve Nyemba b92055f018 bug fix: disk - nujson, refactor 2023-01-04 09:12:14 -06:00
Steve Nyemba 98c9348f9c bug fix: mongodb read 2022-12-14 16:48:40 -06:00
Steve Nyemba 1c254eb133 bug fixes, enhancements mongodb 2022-12-09 16:19:39 -06:00
Steve Nyemba b8fb538ec7 bug fix mongodb,bigquery 2022-11-13 15:45:21 -06:00
Steve Nyemba 883a6ef22f bug fix & new feature 2022-09-19 10:01:34 -05:00
Steve Nyemba 3a3946c7d8 bug fix 2022-08-19 15:32:35 -05:00
Steve Nyemba e5f3b19336 bug fix: added chunks and enhacement with global variables (constants) 2022-08-10 09:17:30 -05:00
Steve Nyemba 245b319e7b added partitioning and chunking to support healthy ETL jobs or write functions 2022-08-03 12:07:27 -05:00
Steve Nyemba 8a6beae956 bug fix: sql handler ... 2022-06-26 15:48:29 -05:00
Steve Nyemba ae1a4cec73 bug fix: removing print statements 2022-06-26 12:18:12 -05:00
Steve Nyemba b0c6bb6ca9 bug fix: environment variable on default database 2022-06-25 14:00:22 -05:00
Steve Nyemba 8ca04e8c48 bug fix: environment variable on default database 2022-06-25 13:59:49 -05:00
Steve Nyemba d7c9e41095 bug fix: meta/sql.py 2022-06-12 21:14:46 -05:00
Steve Nyemba 9805264f87 bug fix: meta data 2022-06-12 20:05:03 -05:00
Steve Nyemba 6b78a82e99 bug fix 2022-06-10 10:27:44 -05:00
Steve Nyemba b2a224f4a7 bug fix: sql lock for parallel processing 2022-06-10 10:11:37 -05:00
Steve Nyemba 40bcfc40f9 bug fix: authentication 2022-05-16 13:14:29 -05:00
Steve Nyemba 7c328e5106 bug fix: mongodb mechanism parameter for authentication 2022-05-16 11:59:46 -05:00
Steve Nyemba db1496dd38 bug fixes 2022-05-16 11:54:28 -05:00
Steve Nyemba f64b945245 new: authentication using file on disk, misc bug fixes 2022-05-16 11:27:36 -05:00
Steve Nyemba 8cd34d902a bug fix: ETL logging and rabbitmq-server listener 2022-05-11 11:17:27 -05:00
Steve Nyemba 67cb7de861 bug fix: logging with ETL 2022-05-06 14:30:59 -05:00
Steve Nyemba 899339f11f bug fix: authentication mongodb 2022-05-06 14:25:12 -05:00
Steve Nyemba fd9523e99a enhancement mongodb 2022-04-29 11:15:32 -05:00
Steve Nyemba e9a6779a9b new file and documentation 2022-04-11 23:31:47 -05:00
Steve Nyemba 98eaa99820 bug fixes and enhancements, added console output(for logging) 2022-04-11 18:34:32 -05:00
Steve Nyemba 6c406407b2 bug fix: ... minor update sqlalchemy 2022-03-31 17:13:24 -05:00
Steve Nyemba e5fadc64a0 optimizations mongodb 2022-03-19 00:02:53 -05:00
Steve Nyemba 38e1bce6c2 bug fixes and optimizations 2022-03-12 12:25:29 -06:00
Steve Nyemba 105ff00224 bugfix: ETL multiprocessing 2022-03-07 18:50:29 -06:00
Steve Nyemba f672c04844 bugfix: sqlalchemy 2022-03-07 15:29:03 -06:00
Steve Nyemba cfc683c1b3 bug fix: transport cli and write function for sql 2022-03-07 14:17:27 -06:00
Steve Nyemba fd9442c298 bug fix: write problems, using drivers 2022-03-04 15:00:30 -06:00
Steve Nyemba 14a551e57b Bug fix: sqlalchemy facilities added 2022-03-03 16:08:24 -06:00
Steve Nyemba a7df7bfbce bug fix: has sql 2022-03-02 13:15:35 -06:00
Steve Nyemba b239a5149f bug fixes and simplifying interfaces 2022-01-29 11:15:45 -06:00
Steve Nyemba ccc05acc01 bug fix: etl engine, sqlite inserts 2021-12-09 15:25:58 -06:00
Steve Nyemba 2de6e51bdb bug fix: simplyfying factory interface 2021-11-18 15:21:26 -06:00
Steve Nyemba 44ef71eb92 bug fix 2021-09-03 01:25:45 -05:00
Steve Nyemba 0c0eaf7063 bug fix: imports, testing code removed 2021-09-01 02:33:40 -05:00
Steve Nyemba 83cc1778c7 bug fix: print 2021-07-28 22:51:19 -05:00
Steve Nyemba 10adde7a08 bug fix: sqlwriter 2021-07-28 22:32:07 -05:00
Steve Nyemba 79cdc0c0d0 bug fix, enhancement with pandas 2021-07-23 15:22:23 -05:00
Steve Nyemba 835e1253a5 bug fix: filereader 2021-07-19 19:07:06 -05:00
Steve Nyemba b21ddd6945 bug fix: removing testing code 2021-07-19 18:54:37 -05:00
Steve Nyemba 19ab510125 cli etl tool 2021-07-08 17:31:29 -05:00
Steve Nyemba 7c2e945996 bug fix: support for netezza 2021-06-29 15:12:28 -05:00
Steve Nyemba 4c810fe57a bug fix 2021-06-20 18:56:12 -05:00
Steve Nyemba b5eb5e0991 bug fix: adding types to read function 2021-06-15 12:30:48 -05:00
Steve Nyemba 91434ec32d bug fix: pandas dtypes 2021-06-14 14:31:30 -05:00
Steve Nyemba cebc784494 bug fix: lock, parallel processing bigquery 2021-04-13 17:29:54 -05:00
Steve Nyemba 84e212d5a3 bug fix: lock, parallel processing bigquery 2021-04-13 17:27:23 -05:00
Steve Nyemba afcc5ed690 bug fix ... 2021-03-29 23:30:54 -05:00
Steve Nyemba 687ffec215 bq support 2021-03-15 12:38:54 -05:00