Where can I find pipeline errors?
A pipeline can fail either because of a node inside a pipeline has completed on an error or because the pipeline definition has caused errors.
Each node in a pipeline is an execution, Task, or deployment. You can either click on the pipeline graph to choose any of the nodes or view their logs for any errors.
You can find pipeline-level errors under the Log tab in the pipeline view.
- Node “name” transitioned to crashed: One of your executions has failed. Check the pipelines executions for an execution that has been completed on an error and check its logs.
- stopping due to 1 incompletable edges: The node couldn’t start because it did not have all the inputs that are required for that step. Review your pipeline edges and make sure you’re passing all the right files.
How can I override default parameters in a pipeline?
You can override the default parameters of a node inside a pipeline using the override attribute.
In the example below we’re creating a pipeline that runs a preprocessing step and three training steps.
The three training nodes are overriding either the default input data, the parameters, or both.
- pipeline: name: Three-Trainings Pipeline nodes: - name: preprocess type: execution step: preprocess-dataset - name: train1 type: execution step: train-model override: inputs: - name: dataset - name: train2 type: execution step: train-model override: inputs: - name: dataset parameters: - name: epochs default: 10 type: integer - name: learning_rate default: 0.001 type: float - name: train3 type: execution step: train-model override: inputs: - name: dataset parameters: - name: epochs default: 15 type: integer - name: learning_rate default: 0.001 type: float edges: - [preprocess.output.preprocessed_mnist.npz, train1.input.dataset] - [preprocess.output.preprocessed_mnist.npz, train2.input.dataset] - [preprocess.output.preprocessed_mnist.npz, train3.input.dataset]