Hadoop on Azure labs

%3CLINGO-SUB%20id%3D%22lingo-sub-1776732%22%20slang%3D%22en-US%22%3EHadoop%20on%20Azure%20labs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1776732%22%20slang%3D%22en-US%22%3E%3CP%3EHello-%3C%2FP%3E%3CP%3EI'm%20working%20on%20installing%20Hadoop%20on%20the%20Azure%20lab%20environment.%20I%20have%20followed%20these%20steps%3A%3CA%20href%3D%22https%3A%2F%2Fkontext.tech%2Fcolumn%2Fhadoop%2F377%2Flatest-hadoop-321-installation-on-windows-10-step-by-step-guide%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fkontext.tech%2Fcolumn%2Fhadoop%2F377%2Flatest-hadoop-321-installation-on-windows-10-step-by-step-guide%3C%2FA%3E%26nbsp%3Band%20am%20using%20this%20template%3A%26nbsp%3B%3C%2FP%3E%3CDIV%20class%3D%22ms-StackItem%20css-67%22%3E%3CDIV%20class%3D%22template__image%22%3EWindows%2010%20Pro%2C%20Version%201909%3C%2FDIV%3E%3C%2FDIV%3E%3CDIV%20class%3D%22ms-StackItem%20css-67%22%3E%3CDIV%3EMedium%3CSPAN%20class%3D%22template__size-divider%22%3E%7C%3C%2FSPAN%3E4%20cores%3CSPAN%20class%3D%22template__size-divider%22%3E%7C%3C%2FSPAN%3E7GB%20RAM%3C%2FDIV%3E%3C%2FDIV%3E%3CDIV%20class%3D%22ms-StackItem%20css-67%22%3E%3CDIV%3EI%20keep%20getting%20an%20error%20when%20I%20try%20to%20format%20the%20namenode%20and%20I%20can't%20figure%20it%20out.%20I'm%20wondering%20if%20anyone%20has%20had%20success%20installing%20hadoop%20on%20Azure%20lab.%20This%20is%20the%20error%20I'm%20getting%3C%2FDIV%3E%3CDIV%3E%3CDIV%20class%3D%22mceNonEditable%20lia-copypaste-placeholder%22%3E%26nbsp%3B%3C%2FDIV%3E%3CP%3E2020-10-13%2018%3A07%3A46%2C044%20INFO%20util.GSet%3A%20VM%20type%20%3D%2064-bit%3CBR%20%2F%3E2020-10-13%2018%3A07%3A46%2C046%20INFO%20util.GSet%3A%200.029999999329447746%25%20max%20memory%20889%20MB%20%3D%20273.1%20KB%3CBR%20%2F%3E2020-10-13%2018%3A07%3A46%2C046%20INFO%20util.GSet%3A%20capacity%20%3D%202%5E15%20%3D%2032768%20entries%3CBR%20%2F%3E2020-10-13%2018%3A07%3A46%2C220%20INFO%20namenode.FSImage%3A%20Allocated%20new%20BlockPoolId%3A%20BP-2025207759-10.140.18.4-1602612466198%3CBR%20%2F%3E2020-10-13%2018%3A07%3A46%2C375%20WARN%20namenode.NameNode%3A%20Encountered%20exception%20during%20format%3A%3CBR%20%2F%3EExitCodeException%20exitCode%3D-1073741515%3A%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell.runCommand(Shell.java%3A1008)%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell.run(Shell.java%3A901)%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell%24ShellCommandExecutor.execute(Shell.java%3A1213)%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell.execCommand(Shell.java%3A1307)%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell.execCommand(Shell.java%3A1289)%3CBR%20%2F%3Eat%20org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java%3A1350)%3CBR%20%2F%3Eat%20org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java%3A1341)%3CBR%20%2F%3Eat%20org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java%3A1294)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.common.Storage%24StorageDirectory.clearDirectory(Storage.java%3A456)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java%3A591)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java%3A613)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java%3A188)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java%3A1206)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java%3A1649)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java%3A1759)%3CBR%20%2F%3E2020-10-13%2018%3A07%3A46%2C386%20ERROR%20namenode.NameNode%3A%20Failed%20to%20start%20namenode.%3CBR%20%2F%3EExitCodeException%20exitCode%3D-1073741515%3A%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell.runCommand(Shell.java%3A1008)%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell.run(Shell.java%3A901)%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell%24ShellCommandExecutor.execute(Shell.java%3A1213)%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell.execCommand(Shell.java%3A1307)%3CBR%20%2F%3Eat%20org.apache.hadoop.util.Shell.execCommand(Shell.java%3A1289)%3CBR%20%2F%3Eat%20org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java%3A1350)%3CBR%20%2F%3Eat%20org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java%3A1341)%3CBR%20%2F%3Eat%20org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java%3A1294)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.common.Storage%24StorageDirectory.clearDirectory(Storage.java%3A456)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java%3A591)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java%3A613)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java%3A188)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java%3A1206)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java%3A1649)%3CBR%20%2F%3Eat%20org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java%3A1759)%3CBR%20%2F%3E2020-10-13%2018%3A07%3A46%2C406%20INFO%20util.ExitUtil%3A%20Exiting%20with%20status%201%3A%20ExitCodeException%20exitCode%3D-1073741515%3A%3CBR%20%2F%3E2020-10-13%2018%3A07%3A46%2C418%20INFO%20namenode.NameNode%3A%20SHUTDOWN_MSG%3A%3CBR%20%2F%3E%2F************************************************************%3CBR%20%2F%3ESHUTDOWN_MSG%3A%20Shutting%20down%20NameNode%20at%20ML-RefVm-145321%2F10.140.18.4%3CBR%20%2F%3E************************************************************%2F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FDIV%3E%3CDIV%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%3E%26nbsp%3B%3C%2FDIV%3E%3C%2FDIV%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
New Contributor

Hello-

I'm working on installing Hadoop on the Azure lab environment. I have followed these steps:https://kontext.tech/column/hadoop/377/latest-hadoop-321-installation-on-windows-10-step-by-step-gui... and am using this template: 

Windows 10 Pro, Version 1909
Medium|4 cores|7GB RAM
I keep getting an error when I try to format the namenode and I can't figure it out. I'm wondering if anyone has had success installing hadoop on Azure lab. This is the error I'm getting
 

2020-10-13 18:07:46,044 INFO util.GSet: VM type = 64-bit
2020-10-13 18:07:46,046 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB
2020-10-13 18:07:46,046 INFO util.GSet: capacity = 2^15 = 32768 entries
2020-10-13 18:07:46,220 INFO namenode.FSImage: Allocated new BlockPoolId: BP-2025207759-10.140.18.4-1602612466198
2020-10-13 18:07:46,375 WARN namenode.NameNode: Encountered exception during format:
ExitCodeException exitCode=-1073741515:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)
at org.apache.hadoop.util.Shell.run(Shell.java:901)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289)
at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1350)
at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:1341)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:1294)
at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:456)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613)
at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1649)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)
2020-10-13 18:07:46,386 ERROR namenode.NameNode: Failed to start namenode.
ExitCodeException exitCode=-1073741515:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)
at org.apache.hadoop.util.Shell.run(Shell.java:901)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1307)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:1289)
at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1350)
at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:1341)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:1294)
at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:456)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591)
at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613)
at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188)
at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1649)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)
2020-10-13 18:07:46,406 INFO util.ExitUtil: Exiting with status 1: ExitCodeException exitCode=-1073741515:
2020-10-13 18:07:46,418 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ML-RefVm-145321/10.140.18.4
************************************************************/

 

 

 
 
 

 

2 Replies

@gina_cooper - we'll follow up with you directly on this to better understand your scenario.

 

Thanks,

Nicole

@nicolehaugen I got it to work! If any schools need to install Hadoop on the Azure lab environment and are having trouble, I wrote down my steps and would be glad to help. It's a pain but it is working now!