Delete all files / folders in subdirectories with a certain pattern on terminal

Today I had a problem on terminal. I have such a folder structure:

./<level_A>/<level_B>_good

or

./<level_A>/<level_B>_bad

Thus I have some folders(level_A) and in each folder there are another level of folders(level_B). On level_B, there are “good” folders and “bad” folders. “Good” folders end with “_good” and “bad” folders end with “_bad”. Also, there are files in all level_B folders.

For example, a good folder can be

./folder_A_1/folder_a_good

and a bad folder can be in the same directory on level A

./folder_A_1/folder_b_bad

Normally, when we want to delete a group of folders that ends with a particular name, we just use

rm -r *_bad

However, this approach does not apply here because I have too many folders on level_A. If I use the rm command, I have to enter and exit all the folders such as

cd folder_A_1

rm -r *_bad

cd ../folder_A_2

rm -r *_bad

...

I have too many folders on level A: folder_A_1 up to folder_A_1000. Hence, I can not use this common approach.

Then I thought of another command I have used: find. That command lists everything in the directory recursively so it is able to target the folders in the subdirectories. That is good. However, how to find, filter and delete? That means, find all the folders recursively, select those paths ending with “_bad”, and delete them. Fortunately, by looking at the manual of find, it turns out that we have a very nice option -delete. Thus we can find all the directories ending with “_bad” using this command

find . -type d -name "*_bad"

-type d requires the target to be a directory. -name “*_bad” specifies the pattern of the name. By this, we find and filtered to target all the directories and subdirectories that ends with “_bad”.

Next, we will make use of the command -delete. 

find . -type d -name "*_bad" -delete

That deleted all the “bad” subdirectories for me and solved my problem. Yay!

I think this can be generalized as a way to simplify processing the system files on terminal. Usually, when it comes to complicated group processing, we have to write bash scripts. However, if we can make good use of the built-in commands, that can save our time.

Copy a folder of files excluding certain files on Mac terminal

Say I want to copy a folder with 1000 files and lots of subfolders, and I need to do this on Mac terminal because it is a part of streamlined process programmed on bash script. Thus I can not just open Finder, select all and copy from there. I can do this very easily with

cp -r source_folder target_path

However, what if I want to exclude some files/subdirectories in this code?

So far, I have put the following requirements:

  • There are lots of files to copy, like 1000.
  • There are still lots of files to exclude, like 50.
  • It must be programmed in bash so no Finder / manual work. One command done

Unfortunately, cp does not offer the function to exclude certain files. Here we need to use rsync.

According to the manual of rsync, it has the following features

  1. It basically copies, like cp, or rcp (remote copy)
  2. It has a lot more options than the basic copy.
  3. It is much faster.

Thus it is good to use rsync. Now let’s go back to the topic: excluding some files or subdirectories during copy. We need to use –exclude option.

We can either put a specific file name or something more general with the asterisk. For example, this one below exclude all hidden files or directories from the copying.

rsync -r source_folder target_path --exclude='.*'

Also, we should notice that, just like cp, copying folders using rsync should put -r. If we only want to exclude a folder called bin, we then should

rsync -r source_folder target_path --exclude='bin'

Excluding multiple elements

If we have more than one element to exclude, we just add more –exclude.

rsync -r source_folder target_path --exclude='bin' --exclude='scripts'

Excluding only hidden folders?

Now that we successfully excluded all hidden files or certain files. What if I still want the files but not the folders? In this case, we use a slash ‘/’ to indicate folders. Like the command below, .*/ means all hidden folders.

rsync -r source_folder target_path --exclude='.*/'

Include back certain files/folders

Say I want to exclude all hidden files but not the hidden folders. However, .*/ means only folders; .* means both files and folders. I did not find anyway to mean only files. Thus, we should use the –include option to include the folders first and then exclude everything, like the command below. To better understand this, according to the manual of rsync, –include tells rsync “not to exclude a pattern”.

rsync -r source_folder target_path --include='.*/' --exclude='.*'