Spark Trouble Shooting - java.net.UnknownHostException

ERROR

If you are sure the hostname is correct, however the NameNode is in HA mode and eventLogging is enabled, you may see this error:

ERROR SparkContext: Error initializing SparkContext.
java.lang.IllegalArgumentException: java.net.UnknownHostException: myhostname
at org.apache.hadoop.security.SecurityUtil.buildTokenService

Solution

A temporary solution is to disable eventLogging, in $SPARK_CONF_DIR/spark-defaults.conf

spark.eventLog.enabled           false