Hadoop on Azure labs

New Contributor

Hello-

I'm working on installing Hadoop on the Azure lab environment. I have followed these steps:https://kontext.tech/column/hadoop/377/latest-hadoop-321-installation-on-windows-10-step-by-step-gui... and am using this template: 

Windows 10 Pro, Version 1909
Medium|4 cores|7GB RAM
I keep getting an error when I try to format the namenode and I can't figure it out. I'm wondering if anyone has had success installing hadoop on Azure lab. This is the error I'm getting
 

2020-10-13 18:07:46,044 INFO util.GSet: VM type = 64-bit
2020-10-13 18:07:46,046 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB
2020-10-13 18:07:46,046 INFO util.GSet: capacity = 2^15 = 32768 entries
2020-10-13 18:07:46,220 INFO namenode.FSImage: Allocated new BlockPoolId: BP-2025207759-10.140.18.4-1602612466198
2020-10-13 18:07:46,375 WARN namenode.NameNode: Encountered exception during format:
ExitCodeException exitCode=-1073741515:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)
at org.apache.hadoop.util.Shell.run(Shell.java:901)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289)
at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1350)
at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:1341)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:1294)
at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:456)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613)
at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1649)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)
2020-10-13 18:07:46,386 ERROR namenode.NameNode: Failed to start namenode.
ExitCodeException exitCode=-1073741515:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)
at org.apache.hadoop.util.Shell.run(Shell.java:901)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289)
at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1350)
at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:1341)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:1294)
at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:456)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613)
at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1649)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)
2020-10-13 18:07:46,406 INFO util.ExitUtil: Exiting with status 1: ExitCodeException exitCode=-1073741515:
2020-10-13 18:07:46,418 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ML-RefVm-145321/10.140.18.4
************************************************************/

 

 

 
 
 

 

2 Replies

@gina_cooper - we'll follow up with you directly on this to better understand your scenario.

 

Thanks,

Nicole

@nicolehaugen I got it to work! If any schools need to install Hadoop on the Azure lab environment and are having trouble, I wrote down my steps and would be glad to help. It's a pain but it is working now!