shreyas-goenka
|
207777849b
|
Log latest error event on pipeline run fail (#239)
DAB config used to test this:
bundle.yml
```
workspace:
host: <deco-azure-prod>
bundle:
name: deco-538
resources:
pipelines:
foo:
name: "[${bundle.name}] log pipeline errors"
libraries:
- notebook:
path: ./myNb.py
development: true
```
myNb.py
```
# Databricks notebook source
print(1/0)
```
Before:
```
2023/03/09 01:28:44 [INFO] [pipelines.foo] Update available at ***
2023/03/09 01:28:44 [INFO] [pipelines.foo] Update status: CREATED
2023/03/09 01:28:46 [INFO] [pipelines.foo] Update status: INITIALIZING
2023/03/09 01:28:52 [INFO] [pipelines.foo] Update status: FAILED
2023/03/09 01:28:52 [INFO] [pipelines.foo] Update has failed!
Error: update failed
```
Now:
```
2023/03/09 01:29:31 [INFO] [pipelines.foo] Update available at ***
2023/03/09 01:29:31 [INFO] [pipelines.foo] Update status: CREATED
2023/03/09 01:29:33 [INFO] [pipelines.foo] Update status: INITIALIZING
2023/03/09 01:29:40 [INFO] [pipelines.foo] Update status: FAILED
2023/03/09 01:29:40 [INFO] [pipelines.foo] Update has failed!
2023/03/09 01:29:40 [ERROR] [pipelines.foo] Update 27bc77 is FAILED.
trace for most recent exception:
Failed to execute python command for notebook '/Users/shreyas.goenka@databricks.com/.bundle/deco-538/default/files/myNb' with id RunnableCommandId(9070319781942164851) and error AnsiResult(---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<command--1> in <cell line: 1>()
----> 1 print(1/0)
ZeroDivisionError: division by zero,Map(),Map(),List(),List(),Map())
Error: update failed
```
|
2023-03-16 12:23:46 +01:00 |
Pieter Noordhuis