shreyas-goenka
|
b4a30c641c
|
Add progress logging for pipeline runs (#283)
Add progress logging for pipeline runs
|
2023-03-31 17:04:12 +02:00 |
|
shreyas-goenka
|
047a189c1e
|
Add job run output logging (#260)
This PR adds output logging for job runs
Tested using unit tests and manually
|
2023-03-21 16:25:18 +01:00 |
Pieter Noordhuis
|
ad666ff796
|
Use new logger throughout codebase (#256)
|
2023-03-17 15:17:31 +01:00 |
|
shreyas-goenka
|
207777849b
|
Log latest error event on pipeline run fail (#239)
DAB config used to test this:
bundle.yml
```
workspace:
host: <deco-azure-prod>
bundle:
name: deco-538
resources:
pipelines:
foo:
name: "[${bundle.name}] log pipeline errors"
libraries:
- notebook:
path: ./myNb.py
development: true
```
myNb.py
```
# Databricks notebook source
print(1/0)
```
Before:
```
2023/03/09 01:28:44 [INFO] [pipelines.foo] Update available at ***
2023/03/09 01:28:44 [INFO] [pipelines.foo] Update status: CREATED
2023/03/09 01:28:46 [INFO] [pipelines.foo] Update status: INITIALIZING
2023/03/09 01:28:52 [INFO] [pipelines.foo] Update status: FAILED
2023/03/09 01:28:52 [INFO] [pipelines.foo] Update has failed!
Error: update failed
```
Now:
```
2023/03/09 01:29:31 [INFO] [pipelines.foo] Update available at ***
2023/03/09 01:29:31 [INFO] [pipelines.foo] Update status: CREATED
2023/03/09 01:29:33 [INFO] [pipelines.foo] Update status: INITIALIZING
2023/03/09 01:29:40 [INFO] [pipelines.foo] Update status: FAILED
2023/03/09 01:29:40 [INFO] [pipelines.foo] Update has failed!
2023/03/09 01:29:40 [ERROR] [pipelines.foo] Update 27bc77 is FAILED.
trace for most recent exception:
Failed to execute python command for notebook '/Users/shreyas.goenka@databricks.com/.bundle/deco-538/default/files/myNb' with id RunnableCommandId(9070319781942164851) and error AnsiResult(---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<command--1> in <cell line: 1>()
----> 1 print(1/0)
ZeroDivisionError: division by zero,Map(),Map(),List(),List(),Map())
Error: update failed
```
|
2023-03-16 12:23:46 +01:00 |
|
Pieter Noordhuis
|
8f4461904b
|
Define flags for running jobs and pipelines (#146)
|
2022-12-23 15:17:16 +01:00 |
|
Pieter Noordhuis
|
7f83463ca3
|
Bump SDK to latest (#151)
|
2022-12-22 09:46:17 +01:00 |
|
Pieter Noordhuis
|
b111416fe5
|
Add `bricks bundle run` command (#134)
|
2022-12-15 15:12:47 +01:00 |