Scribd
Upload
19 views

HDFS Commands - Revised

Uploaded by

nnkumar.mca
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
19 views

HDFS Commands - Revised

Uploaded by

nnkumar.mca
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

JeeraTechno

HDFS COMMANDS

1. copyFromLocal(Copy a file or directory from Local to HDFS)


hadoop fs –copyFromLocal /home/training/dvs/f1.txt dvs_hdfs
Note: Can be used for copying multiple files, similar pattern
files, all the files, a directory

2. moveFromLocal(move a file or directory from Local to HDFS)


Hadoop fs –moveFromLocal /home/training/dvs/f1.txt
dvs_hdfs

3. copyToLocal(Copy a file or directory from HDFS to Local)


hadoop fs –copyToLocal dvs_hdfs/f1.txt /home/training/dvs

4. moveToLocal(Not yet implemented)

5. cp (copy a file from one location to another location inside


HDFS)
hadoop fs –cp dvs_hdfs/file1 dvs_hdfs1

6. mv (move a file from one location to another location inside


HDFS)
hadoop fs –mv dvs_hdfs/file1dvs_hdfs1

7. put (Similar to copyFromLocal)


hadoopfs –put /home/training/dvs/file1dvs_hdfs

8. get (Similar to copyToLocal)


hadoopfs –get dvs_hdfs/file1 /home/training/dvs

2nd Floor, Alpha Block, Sigma SoftTech Park, Varthur Hobli, Bangalore, Karnataka, India 560066.
Facebook : /Jeeratechno | Instagram : /Jeeratechno | Mob: +91 90141 28319,
JeeraTechno
9. getmerge (writes multiple file contents in to a single file in
Local File system)
hadoop fs –getmerge dvs_hdfs/file1 dvs_hdfs/file2
/home/training/dvs/f3
Note : This command is not supported in this version of
Hadoop.

10. mkdir (Create a directory)


hadoop fs –mkdir dvs_hdfs

11. touchz ( can create n no: of empty files in HDFS)


hadoop fs –touchz dvs_hdfs/file1

12. rm (Remove a file)


hadoop fs –rm dvs_hdfs/file1

13. rmr (Can be used for removing a file or Directory


recursively)
hadoop fs –rmr dvs_hdfs/file
hadoop fs –rmr dvs_hdfs/Dir1
Note: Can be used to remove similar pattern files(*.sh, *.txt
etc), all the files(*)

14. ls (Lists all the files & directories)


hadoop fs –ls dvs_hdfs
ls|tail –n (Tail option with List)
hadoop fs –ls dvs_hdfs|tail -10

15. ls|head –n (head option with List)


2nd Floor, Alpha Block, Sigma SoftTech Park, Varthur Hobli, Bangalore, Karnataka, India 560066.
Facebook : /Jeeratechno | Instagram : /Jeeratechno | Mob: +91 90141 28319,
JeeraTechno
hadoop fs –ls dvs_hdfs|head -10

16. cat (Displays the content of a file)


hadoop fs -cat dvs_hdfs/file

17. text(Displays the content of zipped files)


hadoop fs -text dvs_hdfs/file.gz

18. cat|tail –n (Display bottom n lines of a file)


hadoop fs -cat dvs_hdfs/file|tail 10

19. cat|head–n (Display top n lines of a file)


hadoop fs -cat dvs_hdfs/file|head 10

20. cat|wc –l (Counts the no:of lines in a file)


hadoop fs -cat dvs_hdfs/file1|wc –l

21. cat|wc –w (Counts the no:of words in a file)


hadoop fs –catdvs_hdfs/file1|wc –w

22. cat|wc –c (Counts the no:of Characters in a file)


hadoop fs -cat dvs_hdfs/file1|wc –c

23. du (Disk Usage of a file or directory)


hadoop fs –du dvs_hdfs/file1.txt

24. du –h (formats & shows file or directory size in human


readable format)
hadoop fs –dur -hdvs_hdfs
2nd Floor, Alpha Block, Sigma SoftTech Park, Varthur Hobli, Bangalore, Karnataka, India 560066.
Facebook : /Jeeratechno | Instagram : /Jeeratechno | Mob: +91 90141 28319,
JeeraTechno

25. du –s(shows summary of the directories instead of each


file)
hadoop fs –du –s dvs_hdfs

26. df (Disk usage of the entire file system)


hadoop fs –df
O/P:
Filesystem Size Used Available Use%
hdfs://nameservice1 328040332591104 102783556870823
210750795833344 31%

27. df –h (Formats & shows in the human readable format)


hadoop fs -df –h
O/P:
Filesystem Size Used Available Use%
hdfs://nameservice1 298.4 T 93.5 T 191.7 T 31%

28. count(Counts all the Directories & Files in the given path)
hadoop fs –count dvs_hdfs

29. fsck(To check file system health)


Hadoop fsck dvs_hdfs

30. fsck –files –blocks (Displays corresponding Files& their


block level info)
Hadoop fsck dvs_hdfs/file1.txt –files -blocks
2nd Floor, Alpha Block, Sigma SoftTech Park, Varthur Hobli, Bangalore, Karnataka, India 560066.
Facebook : /Jeeratechno | Instagram : /Jeeratechno | Mob: +91 90141 28319,
JeeraTechno

31. fsck –files –blocks –locations (Displays files& block level


info including the block location)
32. hadoopfsckdvs_hdfs/f1.txt –files –blocks –locations -racks

33. setrep(used to change the replication factor a file or a


directory)
hadoop fs –setrep 5 dvs_hdfs/file1
Hadoop fs –setrep 5 –w dvs_hdfs/ABC
-w It requests that the command waits for the replication to
complete. This
can potentially take a very long time.

34. Controlling block size at file level without changing the


block size in hdfs-site.xml
Hadoop fs –D dfs.block.size=134217728 –put
source_pathdestination_path

35. Controlling replication at file level irrespective of the


default replication set to 3
Hadoop fs –D dfs.replication=2 –put
source_pathdestination_path
36. Setting replication factor for a directory in HDFS
Hadoop fs –setrep 5 –R dvs_hdfs/ABC
Note: All the files copied under this directory will be having a
replication factor of 5 irrespective of the default replication set.
37. Safe Mode
Hadoop dfsadmin –safemode leave
Hadoop dfsadmin –safemode enter
2nd Floor, Alpha Block, Sigma SoftTech Park, Varthur Hobli, Bangalore, Karnataka, India 560066.
Facebook : /Jeeratechno | Instagram : /Jeeratechno | Mob: +91 90141 28319,
JeeraTechno
Hadoop dfsadmin –safemode get
38. Delete all the files in trash
hadoop fs -expunge
39. Copying a file from one cluster to another cluster
hadoop fs -distcp hdfs://namenodeA/emp.csv
hdfs://namenodeB/
40.

2nd Floor, Alpha Block, Sigma SoftTech Park, Varthur Hobli, Bangalore, Karnataka, India 560066.
Facebook : /Jeeratechno | Instagram : /Jeeratechno | Mob: +91 90141 28319,

You might also like