Bash Script With Multiple Papermill Commands Does Not Fail On Notebook Errors

I have a refresh_data.sh file which contains multiple papermill commands, for example: papermill notebook_1.ipynb output_1.ipynb -p start '2017-12-01' -p date '2017-12-31' papermil

Solution 1:

If your bash script is configured with: set -e it will fail if a command errors out:

Automatic exit from bash shell script on error

#!/bin/bashset -e
# Any subsequent(*) commands which fail will cause the shell script to exit immediately

You can run papermill using:

--log-output to get more information about why your notebook fail.

papermill "${INPUT_NOTEBOOK_PATH}""${OUTPUT_NOTEBOOK_PATH}" --log-output

To capture notebook execution result you can always capture the result of any previous command using $?:

  papermill "${INPUT_NOTEBOOK_PATH}""${OUTPUT_NOTEBOOK_PATH}" --log-output
  notebook_result=$?
  if [[ ${notebook_result} -eq 0 ]]; thenecho"All good"elseecho$notebook_resultfi