Solved — Apache Spark driverState FAILED when using REST API

Recently I encountered a continuos error when submitting a new job to Apache Sprak via the REST API. All jobs submissions happen successfully and return a good result in first instance.

{
  "action" : "CreateSubmissionResponse",
  "message" : "Driver successfully submitted as driver-20240427093303-0006",
  "serverSparkVersion" : "3.5.1",
  "submissionId" …