4 Replies Latest reply: Nov 17, 2012 4:00 PM by Catch~22 RSS

    Copy a lot of files to external hard

    Saeed Hassanpour
      Dear Sir/Madam,

      How can I copy a lot of folders and files the contents of *“image & voice & text”* to external hard or another machine quickly?
      In addition,it can be the server is OEL 4.8/OEL 5.4/OEL 6.3.

      Best regards,
      Saeed
        • 1. Re: Copy a lot of files to external hard
          sabre150
          The Devil is in the detail of what servers (FTP, SFTP, SCP, SAMBA, NFS etc etc etc) you have running on the target machine, how many files and how many bytes you want to transfer. Also you need to consider how secure the transfer must be, what action needs to be taken if any of the transfers fails for some reason and how the files to be transferred are to be selected. Without this knowledge it is difficult to advise.
          • 2. Re: Copy a lot of files to external hard
            Catch~22
            I'd probably use the following:

            <pre>
            su - root
            tar -zcf /home/bigfile.tgz /path/to/source/
            scp /home/bigfile.tgz root@ip.remotehost:/home
            </pre>

            The above should work on any RHEL based systems without the need to install or configure any service beside standard networking. SCP uses the same authentication and security as SSH and is usually allowed by network firewalls.
            • 3. Re: Copy a lot of files to external hard
              Saeed Hassanpour
              The size of folder in linux is more than 300 GB,yet one of folder is more than 100 GB the content of a lot of files.
              Also, when I wanted to find out size of this folder with (du -sh *), it took time.In addition, I have two questions.

              1) Is this syntax the best method?(tar -zcf /home/bigfile.tgz /path/to/source/)
              2) How can I see how much time it has left to finish zip file with tar sysntax?

              Regards,
              Saeed
              • 4. Re: Copy a lot of files to external hard
                Catch~22
                Also, when I wanted to find out size of this folder with (du -sh *), it took time.In addition, I have two questions.
                You will probably want to use du -sh /path/directory to get the total of a specific directory. Anyway, the more files, the longer it will take to gather statistics and compute them. The same applies to the copy process. A system is usually faster to tar (archive) a directory on the same hard disk than to copy thousands of individual files to another media. The same applies to copying files via the network. The tar -v option will show you what it is doing.
                1) Is this syntax the best method?(tar -zcf /home/bigfile.tgz /path/to/source/)
                When asking a question about how to copy files, providing information about the size of your folder/files and available file transfer options should be provided. The least information you put into your initial question, the more general or vague will be the answer. There is hardly ever a one fits all solution to any question in Unix/Linux.
                2) How can I see how much time it has left to finish zip file with tar sysntax?
                You can't. The -z option of the tar command will compress the archive, which will slow down the archive creation process, but may result in a substantially smaller file and hence reduce the space it takes to create the archive, and time it takes to copy to another system. Time and compression ratio of your files will depend on type of file.

                If you have an external USB-2 HD, you can also use that to transfer the files to another system. I suggest the following:
                <pre>

                [dude@vm016 ~]$ su - root
                [root@vm016 ~]# yum -y install lsscsi

                Plug in your USB device
                [root@vm016 ~]# lsscsi
                Determine your USB device, e.g. /dev/sdd

                Initialize the USB media: (All data will be erased!)
                [root@vm016 ~]# parted -s */dev/sdd* mklabel msdos mkpart primary 1 -- -1

                Create ext4 filesystem:
                [root@vm016 ~]# mkfs.ext4 -L backup */dev/sdd*

                Mount partition 1 of the USB device and copy the data:
                [root@vm016 ~]# mkdir -p /mnt/usb
                [root@vm016 ~]# mount */dev/sdd1* /mnt/usb
                [root@vm016 ~]# tar -cvf /mnt/usb/archive.tar /path_to/source_directory/
                </pre>

                Edited by: Dude on Nov 17, 2012 1:57 PM