Creating large tar archives

Problem

You need to tar thousands of files into a single tar archive.

Analysis

You can not simply do “tar cvf archive.tar ./*” because tar gives you an error that there are too many files in the argument.

Solution

Instead do the following:

1. Create the archive:

$ tar cvf /tmp/archive.tar somefile

2. Add files to your archive:

$ find . | xargs tar rvf /tmp/archive.tar

This command will add (update the archive) all the files from the current directory (no matter how many tere are) to your archive. It’s fast and efficient. xargs divides the files from STDIN to arguments acceptable for tar.

References

1. http://www.devdaily.com/blog/post/linux-unix/using-find-xargs-tar-create-huge-archive-cygwin-linux-unix/
2. 2.4 How to Add Files to Existing Archives http://www.apl.jhu.edu/Misc/Unix-info/tar/tar_28.html