Commit Graph

1 Commits

Author SHA1 Message Date
Andrew Nester cdea775bd2
Fixed spark version check for clusters defined in the same bundle (#2374)
## Changes
Previously using python wheel tasks in the tasks with compute referering
to interactive cluster defied in the same bundle would produce a warning
like below

```
GET /api/2.1/clusters/get?cluster_id=${resources.clusters.development_cluster.id}
< HTTP/2.0 400 Bad Request
< {
<   "error_code": "INVALID_PARAMETER_VALUE",
<   "message": "Cluster ${resources.clusters.development_cluster.id} does not exist"
< } pid=14465 mutator=seq mutator=initialize mutator=seq mutator=PythonWrapperWarning sdk=true
```

This PR fixes it by making sure that we check spark version for such
clusters based on its bundle configuration and don't make API calls

## Tests
Added acceptance test
2025-02-26 13:04:45 +01:00