Unable to configure log directory of Spark History Server to Storage Blob when deployed on AKS

%3CLINGO-SUB%20id%3D%22lingo-sub-1770360%22%20slang%3D%22en-US%22%3EUnable%20to%20configure%20log%20directory%20of%20Spark%20History%20Server%20to%20Storage%20Blob%20when%20deployed%20on%20AKS%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1770360%22%20slang%3D%22en-US%22%3E%3CP%3EI%26nbsp%3B%3CSPAN%3Eam%20trying%20to%20deploy%20Spark%20History%20Server%20on%20AKS%20and%20wanted%20to%20point%20it's%20log%20directory%20to%20Storage%20Blob.%20For%20achieving%20this%2C%20I%20am%20putting%20my%20configs%20on%20the%26nbsp%3B%3C%2FSPAN%3E%3CSTRONG%3Evalues.yaml%3C%2FSTRONG%3E%3CSPAN%3E%26nbsp%3Bfile%20as%20below%3A-%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%3Ewasbs%3A%0A%20%20enableWASBS%3A%20true%0A%20%20secret%3A%20azure-secrets%0A%20%20sasKeyMode%3A%20false%0A%20%20storageAccountKeyName%3A%20azure-storage-account-key%0A%20%20storageAccountNameKeyName%3A%20azure-storage-account-name%0A%20%20containerKeyName%3A%20%20azure-blob-container-name%0A%20%20logDirectory%3A%20wasbs%3A%2F%2F%2Ftest%2Fpiyush%2Fspark-history%0A%0Apvc%3A%0A%20%20enablePVC%3A%20false%0Anfs%3A%0A%20%20enableExampleNFS%3A%20false%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3EFirst%2C%20I%20am%20creating%20the%20azure-secrets%20using%20the%20below%20command%3A-%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%3Ekubectl%20create%20secret%20generic%20azure-secrets%20--from-file%3Dazure-storage-account-name%20--from-file%3Dazure-blob-container-name%20--from-file%3Dazure-storage-account-key%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3EAfter%20that%2C%20I%20am%20running%20the%20following%20set%20of%20commands%3A-%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%3Ehelm%20repo%20add%20stable%20https%3A%2F%2Fkubernetes-charts.storage.googleapis.com%0Ahelm%20install%20stable%2Fspark-history-server%20--values%20values.yaml%20--generate-name%26nbsp%3B%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%3EBut%20while%20doing%20go%2C%20I%20am%20getting%20the%20following%20error%3A-%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CPRE%3E2020-10-05%2019%3A17%3A56%20INFO%20%20HistoryServer%3A2566%20-%20Started%20daemon%20with%20process%20name%3A%2012%40spark-history-server-1601925447-57c5476fb-5wh6q%0A2020-10-05%2019%3A17%3A56%20INFO%20%20SignalUtils%3A54%20-%20Registered%20signal%20handler%20for%20TERM%0A2020-10-05%2019%3A17%3A56%20INFO%20%20SignalUtils%3A54%20-%20Registered%20signal%20handler%20for%20HUP%0A2020-10-05%2019%3A17%3A56%20INFO%20%20SignalUtils%3A54%20-%20Registered%20signal%20handler%20for%20INT%0A2020-10-05%2019%3A17%3A56%20WARN%20%20NativeCodeLoader%3A62%20-%20Unable%20to%20load%20native-hadoop%20library%20for%20your%20platform...%20using%20builtin-java%20classes%20where%20applicable%0A2020-10-05%2019%3A17%3A56%20INFO%20%20SecurityManager%3A54%20-%20Changing%20view%20acls%20to%3A%20root%0A2020-10-05%2019%3A17%3A56%20INFO%20%20SecurityManager%3A54%20-%20Changing%20modify%20acls%20to%3A%20root%0A2020-10-05%2019%3A17%3A56%20INFO%20%20SecurityManager%3A54%20-%20Changing%20view%20acls%20groups%20to%3A%0A2020-10-05%2019%3A17%3A56%20INFO%20%20SecurityManager%3A54%20-%20Changing%20modify%20acls%20groups%20to%3A%0A2020-10-05%2019%3A17%3A56%20INFO%20%20SecurityManager%3A54%20-%20SecurityManager%3A%20authentication%20disabled%3B%20ui%20acls%20disabled%3B%20users%20%20with%20view%20permissions%3A%20Set(root)%3B%20groups%20with%20view%20permissions%3A%20Set()%3B%20users%20%20with%20modify%20permissions%3A%20Set(root)%3B%20groups%20with%20modify%20permissions%3A%20Set()%0A2020-10-05%2019%3A17%3A56%20INFO%20%20FsHistoryProvider%3A54%20-%20History%20server%20ui%20acls%20disabled%3B%20users%20with%20admin%20permissions%3A%20%3B%20groups%20with%20admin%20permissions%0AException%20in%20thread%20%22main%22%20java.lang.reflect.InvocationTargetException%0A%20%20%20%20%20%20%20%20at%20sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native%20Method)%0A%20%20%20%20%20%20%20%20at%20sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java%3A62)%0A%20%20%20%20%20%20%20%20at%20sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java%3A45)%0A%20%20%20%20%20%20%20%20at%20java.lang.reflect.Constructor.newInstance(Constructor.java%3A423)%0A%20%20%20%20%20%20%20%20at%20org.apache.spark.deploy.history.HistoryServer%24.main(HistoryServer.scala%3A280)%0A%20%20%20%20%20%20%20%20at%20org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)%0ACaused%20by%3A%20java.lang.RuntimeException%3A%20java.lang.ClassNotFoundException%3A%20Class%20org.apache.hadoop.fs.azure.NativeAzureFileSystem%20not%20found%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.conf.Configuration.getClass(Configuration.java%3A2195)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java%3A2654)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java%3A2667)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.fs.FileSystem.access%24200(FileSystem.java%3A94)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.fs.FileSystem%24Cache.getInternal(FileSystem.java%3A2703)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.fs.FileSystem%24Cache.get(FileSystem.java%3A2685)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.fs.FileSystem.get(FileSystem.java%3A373)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.fs.FileSystem.get(FileSystem.java%3A364)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.fs.Path.getFileSystem(Path.java%3A295)%0A%20%20%20%20%20%20%20%20at%20org.apache.spark.deploy.history.FsHistoryProvider.%26lt%3Binit%26gt%3B(FsHistoryProvider.scala%3A117)%0A%20%20%20%20%20%20%20%20at%20org.apache.spark.deploy.history.FsHistoryProvider.%26lt%3Binit%26gt%3B(FsHistoryProvider.scala%3A86)%0A%20%20%20%20%20%20%20%20...%206%20more%0ACaused%20by%3A%20java.lang.ClassNotFoundException%3A%20Class%20org.apache.hadoop.fs.azure.NativeAzureFileSystem%20not%20found%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java%3A2101)%0A%20%20%20%20%20%20%20%20at%20org.apache.hadoop.conf.Configuration.getClass(Configuration.java%3A2193)%0A%20%20%20%20%20%20%20%20...%2016%20more%3C%2FPRE%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAny%20type%20of%20help%20or%20suggestion%20would%20be%20appreciated.%3C%2FP%3E%3CP%3EThanks%20in%20advance!%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ERegards%2C%3C%2FP%3E%3CP%3EPiyush%20Thakur%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1770360%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EKubernetes%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1770581%22%20slang%3D%22en-US%22%3ERe%3A%20Unable%20to%20configure%20log%20directory%20of%20Spark%20History%20Server%20to%20Storage%20Blob%20when%20deployed%20on%20AKS%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1770581%22%20slang%3D%22en-US%22%3EYou%20mean%20I%20need%20to%20change%20or%20rename%20my%20values.yaml%20file%20to%20the%20py.yaml%20file%3F%3C%2FLINGO-BODY%3E
Highlighted
New Contributor

am trying to deploy Spark History Server on AKS and wanted to point it's log directory to Storage Blob. For achieving this, I am putting my configs on the values.yaml file as below:-

 

wasbs:
  enableWASBS: true
  secret: azure-secrets
  sasKeyMode: false
  storageAccountKeyName: azure-storage-account-key
  storageAccountNameKeyName: azure-storage-account-name
  containerKeyName:  azure-blob-container-name
  logDirectory: wasbs:///test/piyush/spark-history

pvc:
  enablePVC: false
nfs:
  enableExampleNFS: false

 

First, I am creating the azure-secrets using the below command:-

 

kubectl create secret generic azure-secrets --from-file=azure-storage-account-name --from-file=azure-blob-container-name --from-file=azure-storage-account-key

 

After that, I am running the following set of commands:-

 

helm repo add stable https://kubernetes-charts.storage.googleapis.com
helm install stable/spark-history-server --values values.yaml --generate-name 

 

But while doing go, I am getting the following error:-

 

2020-10-05 19:17:56 INFO  HistoryServer:2566 - Started daemon with process name: 12@spark-history-server-1601925447-57c5476fb-5wh6q
2020-10-05 19:17:56 INFO  SignalUtils:54 - Registered signal handler for TERM
2020-10-05 19:17:56 INFO  SignalUtils:54 - Registered signal handler for HUP
2020-10-05 19:17:56 INFO  SignalUtils:54 - Registered signal handler for INT
2020-10-05 19:17:56 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2020-10-05 19:17:56 INFO  SecurityManager:54 - Changing view acls to: root
2020-10-05 19:17:56 INFO  SecurityManager:54 - Changing modify acls to: root
2020-10-05 19:17:56 INFO  SecurityManager:54 - Changing view acls groups to:
2020-10-05 19:17:56 INFO  SecurityManager:54 - Changing modify acls groups to:
2020-10-05 19:17:56 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
2020-10-05 19:17:56 INFO  FsHistoryProvider:54 - History server ui acls disabled; users with admin permissions: ; groups with admin permissions
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:280)
        at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azure.NativeAzureFileSystem not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
        at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2654)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2703)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2685)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:373)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:364)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
        at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:117)
        at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:86)
        ... 6 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azure.NativeAzureFileSystem not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193)
        ... 16 more

 

Any type of help or suggestion would be appreciated.

Thanks in advance!

 

Regards,

Piyush Thakur

2 Replies
Highlighted
Highlighted
You mean I need to change or rename my values.yaml file to the py.yaml file?