Where can I find pipeline errors?
A pipeline can fail either because of a node inside a pipeline has completed on an error or because the pipeline definition has caused errors.
Each node in a pipeline is an execution, Task, or deployment. You can either click on the pipeline graph to choose any of the nodes or view their logs for any errors.
You can find pipeline-level errors under the Log tab in the pipeline view.
- Node “name” transitioned to crashed: One of your executions has failed. Check the pipelines executions for an execution that has been completed on an error and check its logs.
- stopping due to 1 incompletable edges: The node couldn’t start because it did not have all the inputs that are required for that step. Review your pipeline edges and make sure you’re passing all the right files.
How can I override default parameters in a pipeline?
You can override the default parameters of a node inside a pipeline using the override attribute.
In the example below we’re creating a pipeline that runs a preprocessing step and three training steps.
The three training nodes are overriding either the default input data, the parameters, or both.
- pipeline:
name: Three-Trainings Pipeline
nodes:
- name: preprocess
type: execution
step: preprocess-dataset
- name: train1
type: execution
step: train-model
override:
inputs:
- name: dataset
- name: train2
type: execution
step: train-model
override:
inputs:
- name: dataset
parameters:
- name: epochs
default: 10
type: integer
- name: learning_rate
default: 0.001
type: float
- name: train3
type: execution
step: train-model
override:
inputs:
- name: dataset
parameters:
- name: epochs
default: 15
type: integer
- name: learning_rate
default: 0.001
type: float
edges:
- [preprocess.output.preprocessed_mnist.npz, train1.input.dataset]
- [preprocess.output.preprocessed_mnist.npz, train2.input.dataset]
- [preprocess.output.preprocessed_mnist.npz, train3.input.dataset]