Ubuntu File System Commands

Xargs

Read a file.txt and create a directory for each line of the file
file.txt contents = (each on a separate line) apple oranges pear
cat file.txt | sort | uniq | xargs -I {} mkdir -p /var/www/fruits/{}
find dir/ -type f -print0 | xargs -0 chmod 755
(print0 is used to make sure the null character will separate them and the -0 make sure xargs uses that null charcter
find . -name "*fruit.txt" -print0 | xargs -0 -I {} cp {} /folder/{}.backup
Find files in the current directory with fruit in the filename "{}"" is the place holder for the filename. Copy the {} to the specified folder
find . -name "*fruit.txt" -depth 1 -print0 | xargs -0 -I {} rm
find . -name "*invoice*" -print0 | xargs -0 grep -li 'outwater' | xargs -I {} cp {} /dir/{}
Find all files with the word invoice then send it to grep to search in the files for the text outwater then copy those files to the dir

Diff Command

diff -y originalfile.txt revisedfile.txt

Cut Command, Can extract contiguious text from a file. eg charcters 2 - 10 of every line

cut -c 2-10 textfile.txt
Will extract characters 2 through 10 on each line
cut -c 2-10,30-35 filename.txt
will extract 2-10 and 30-35
cut -f 2,6 -d "," filename.csv
-f along with the -d option wil allow you to add a delimiter

TR (translate Function)

replace the , in a text file with a ; then pipe it back to the file
tr ',' ';' < somefile.csv > somefile.csv

Standard Input and Standard Output

Send Sorted file to new file
sort somefile.txt > newfilename.txt
To Append use >> insted of >
Supressing Output
ls -la > /dev/null

How To Change Multiple File Extensions From The Terminal

1. Open a new terminal and create the following directory in you desktop.

cd  /home/oltjano/Desktop

mkdir unixmen_tutorial

2.  cd to unixmen_tutorial and create the following files.

a.txt   b.txt  c.txt

3.  Ok guys it is time for some action. Run the following piece of code in the terminal and see what happens.

for i in *.txt; do echo $i; done

4. The following screenshot shows the result  that you should get in your terminal.

So what we are trying to do here is running a for loop and printing every filename with the .txt in the current directory. Ok, now run the following commands. It is used to strip the extension from a file.

a=a.txt

echo ${a/.txt}

5.  Do you see the following result?

6.  Ok, now run the following piece of code in your terminal. Have the file extensions changed?

for i in *.txt;  do mv "$i" "${i/.txt}".jpg; done

Wget Command

Download a single file

$ wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.1.tar.gz

Download and store it with a different name.

$ wget -O taglist.zip http://www.vim.org/scripts/download_script.php?src_id=7701

Download in the Background Using wget -b

For a huge download, put the download in background using wget option -b as shown below.

$ wget -b http://www.openss7.org/repos/tarballs/strx25-0.9.2.1.tar.bz2

Mask User Agent and Display wget like Browser Using wget –user-agent

Some websites can disallow you to download its page by identifying that the user agent is not a browser. So you can mask the user agent by using –user-agent options and show wget like a browser as shown below.

 wget --user-agent="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092416 Firefox/3.0.3" URL-TO-DOWNLOAD

Download Multiple Files / URLs Using Wget -i

First, store all the download files or URLs in a text file as:

$ cat > download-file-list.txt

URL1

URL2

URL3

URL4

Next, give the download-file-list.txt as argument to wget using -i option as shown below.

$ wget -i download-file-list.txt

Download a Full Website Using wget –mirror

Following is the command line which you want to execute when you want to download a full website and made available for local viewing.

$ wget --mirror -p --convert-links -P ./LOCAL-DIR WEBSITE-URL

–mirror : turn on options suitable for mirroring.

-p : download all files that are necessary to properly display a given HTML page.

–convert-links : after the download, convert the links in document for local viewing.

-P ./LOCAL-DIR : save all the files and directories to the specified directory.

Reject Certain File Types while Downloading Using wget –reject

You have found a website which is useful, but don’t want to download the images you can specify the following.

$ wget --reject=gif WEBSITE-TO-BE-DOWNLOADED

Download Only Certain File Types Using wget -r -A

You can use this under following situations:

Download all images from a website

Download all videos from a website

Download all PDF files from a website

$ wget -r -A.pdf http://url-to-webpage-with-pdfs/

FTP Download With wget

You can use wget to perform FTP download as shown below.

Anonymous FTP download using Wget

$ wget ftp-url

FTP download using wget with username and password authentication.

$ wget --ftp-user=USERNAME --ftp-password=PASSWORD DOWNLOAD-URL

Xargs Linux Command

Copy all images to external hard-drive

# ls *.jpg | xargs -n1 -i cp {} /external-hard-drive/directory

Search all jpg images in the system and archive it.

# find / -name *.jpg -type f -print | xargs tar -cvzf images.tar.gz

Download all the URLs mentioned in the url-list.txt file

# cat url-list.txt | xargs wget –c

Move Multiple folders to another directory

mv -v /home/user1/Desktop/folder1/* /var/tmp/

This will move the contents of folder1 to tmp folde

Using Grep and find to search through eml files

Using Grep and Find to search through .eml files for a specific phrase

go to the dir in question

find . -exec grep -ils 'text to find' /dev/null {} \; | xargs -I {} cp -p {} /Users/homedir/Desktop/

above will find the files and copy them to specified folder

find . -exec grep -ils 'text to find\|more text to find\|even more text' /dev/null {} \; | xargs -I {} cp -p {} /Users/homedir/Desktop/

Above will find multiple search strings

find . -type f -name ".DS_Store" -exec rm -f {} \;

Above will find Ds filese in current dir and subdir and delete them

find . -exec grep -ls 'text to find' /dev/null {} \;

find . -exec grep -H 'text to look for {} \;

find . -exec grep -n 'text to look for' /dev/null {} \;

find . -exec grep -n 'yuly' /dev/null {} \; -print >> /Volumes/RAIDset1/1share/text.txt

list files that contain the name "out"

ls -la | grep out

Find files in a dir with a string. the -i is case insensitive -w is exact word

find dirname | grep -i  string

SED find and replace

sed -i (inplace) s (subsitute) /find/replace/ g (global). You can replce the / with any other delimiter eg | or :. eg. if you want to find text "/mac" then you would use : as the delimiter sed -i 's:/mac:mac:g'

Find and Replace Multiple (add the -e switch
sed -e 's/find/replace/g' -e 's/find/replace/g'
Let us start off simple:
Imagine you have a large file ( txt, php, html, anything ) and you want to replace all the words "ugly" with "beautiful"


This is the command:

CODE
sed -i 's/ugly/beautiful/g' /home/bruno/old-friends/sue.txt


"sed" edits "-i in place ( on the spot ) and replaces the word "ugly with "beautiful" in the file "/home/bruno/old-friends/sue.txt"


Imagine you have a whole lot of files in a directory and you want the same command to do all those files in one go 
Remember the find command ? We will combine the two:

CODE
$ find /home/bruno/old-friends -type f -exec sed -i 's/ugly/beautiful/g' {} \;


Sure in combination with the find command you can do all kind of nice tricks, even if you don't remember where the files are located !


Aditionally I did find a little script on the net for if you often have to find and replace multiple files at once:

CODE
#!/bin/bash
     for fl in *.php; do
     mv $fl $fl.old
     sed 's/FINDSTRING/REPLACESTRING/g' $fl.old > $fl
     rm -f $fl.old
     done

just replace the "*.php", "FINDSTRING" and "REPLACESTRING" make it executable and you are set.

I changed a www address in 183 .html files in one go with this little script . . . but note that you have to use "escape-signs" ( \ ) if there are slashes in the text you want to replace, so as an example: 's/www.search.yahoo.com\/images/www.google.com\/linux/g' to change www.search.yahoo.com/images to www.google.com/linux


For the lovers of perl I also found this one:

CODE
# perl -e "s/old_string/new_string/g;" -pi.save $(find DirectoryName -type f)

Creating ISO File from a folder

If you want to make an iso file from a directory containing other files and sub-directories via the terminal, you can use the following command:

mkisofs -o image.iso -R /path/to/folder/

If you wish to backup the home folder, use this command:

mkisofs -o image.iso -R $HOME

Mount ftp server as local drive

1. Installation

First install curlftpfs package. On Debian or Ubuntu it would simple as:

apt-get install curlftpfs

2. Mount ftp directory

What needs to be done next is to create a mount point:

# mkdir /mnt/my_ftp

next use curlftpfs to mount your remote ftp site. Suppose my access credentials are as follows:

username: ftp-user
password: ftp-pass
host/IP: my-ftp-location.local
the actual curlftpfs mount command would be:

# curlftpfs ftp-user:ftp-pass@my-ftp-location.local /mnt/my_ftp/

 

Caution:

ftp uses unencrypted passwords so anyone can intercept your password without much effort. Therefore use curlftpfs in combination with SSL certificates if your are not mounting some local LAN ftp server.

On Debian you can mount ftp using curlftpfs as a root and this allows only root user to access ftp mount.  No other users are allowed since by default only users that mounts has and access to mount directory. When mounting ftp as a non-root user you may get a following error message:

 

fuse: failed to open /dev/fuse: Permission denied

Rather that changing permissions of /dev/fuse you can allow other users to access ftp mount directory with an curlftpfs's option allow_other. The command will look similar to the one below:

# curlftpfs -o allow_other ftp-user:ftp-pass@my-ftp-location.local /mnt/my_ftp/

3. Mount ftp with curlftpfs using /etc/fstab

 

Since we do not want put any passwords to /etc/fstab file we will first create a /root/.netrc file with a ftp username and password using this format:

machine my-ftp-location.local

login ftp-user

password ftp-pass

Now change permissions of this file to 600:

# chmod 600 /root/.netrc

Check uid and gid of your non-root user. This user will have a access to ftp mount directory:

$ id

In the next step add a following line to your /etc/fstab file ( change credentials for your ftp user ):

curlftpfs#my-ftp-location.local /mnt/my_ftp fuse allow_other,uid=1000,gid=1000,umask=0022 0 0

Now mount ftp with:

mount -a

Creating Aliases

2 . You can add more that one commands on a line in terminal if you follow it with a ; (semi-colan) eg cd etc; ls; cd /

will change to the etc dir ls the dir and then cd to root.

 

3. Create a Temorary Alias your cammands. alias foo='cd /etc; ls; cd /'

this will create an alias with the name foo. you can execute foo which will run the command.

- to remove the command: unalias foo

- display commands in an alias: type foo

After shell is closed alias is gone

 

4. Permanent alias: edit the .bashrc file -

nano .bashrc

add your alias to this file. eg. alias foo='echo hello world'

save and close file, then issue command source .bashrc // this will reload the changes in the .bashrc file

For Mac the the .bashrc file is in the Home dir ~ and it's called .bash_profile

Find Command

Part I – Basic Find Commands for Finding Files with Names

1. Find Files Using Name in Current Directory
Find all the files whose name is tecmint.txt in a current working directory.
# find . -name tecmint.txt

./tecmint.txt
2. Find Files Under Home Directory

Find all the files under /home directory with name tecmint.txt.
# find /home -name tecmint.txt

/home/tecmint.txt

3. Find Files Using Name and Ignoring Case

Find all the files whose name is tecmint.txt and contains both capital and small letters in /homedirectory.
# find /home -iname tecmint.txt

./tecmint.txt
./Tecmint.txt

4. Find Directories Using Name

Find all directories whose name is Tecmint in / directory.
# find / -type d -name Tecmint

/Tecmint
5. Find PHP Files Using Name
Find all php files whose name is tecmint.php in a current working directory.
# find . -type f -name tecmint.php

./tecmint.php

6. Find all PHP Files in Directory

Find all php files in a directory.
# find . -type f -name "*.php"

./tecmint.php
./login.php
./index.php
Part II – Find Files Based on their Permissions
7. Find Files With 777 Permissions
Find all the files whose permissions are 777.
# find . -type f -perm 0777 -print

8. Find Files Without 777 Permissions

Find all the files without permission 777.
# find / -type f ! -perm 777

9. Find SGID Files with 644 Permissions

Find all the SGID bit files whose permissions set to 644.
# find / -perm 2644

10. Find Sticky Bit Files with 551 Permissions

Find all the Sticky Bit set files whose permission are 551.
# find / -perm 1551

11. Find SUID Files

Find all SUID set files.
# find / -perm /u=s
12. Find SGID Files
Find all SGID set files.
# find / -perm /g+s
13. Find Read Only Files
Find all Read Only files.
# find / -perm /u=r
14. Find Executable Files
Find all Executable files.
# find / -perm /a=x
15. Find Files with 777 Permissions and Chmod to 644
Find all 777 permission files and use chmod command to set permissions to 644.
# find / -type f -perm 0777 -print -exec chmod 644 {} \;
16. Find Directories with 777 Permissions and Chmod to 755
Find all 777 permission directories and use chmod command to set permissions to 755.
# find / -type d -perm 777 -print -exec chmod 755 {} \;
17. Find and remove single File
To find a single file called tecmint.txt and remove it.
# find . -type f -name "tecmint.txt" -exec rm -f {} \;
18. Find and remove Multiple File
To find and remove multiple files such as .mp3 or .txt, then use.
# find . -type f -name "*.txt" -exec rm -f {} \;

OR

# find . -type f -name "*.mp3" -exec rm -f {} \;
19. Find all Empty Files
To file all empty files under certain path.
# find /tmp -type f -empty
20. Find all Empty Directories
To file all empty directories under certain path.
# find /tmp -type d -empty
21. File all Hidden Files
To find all hidden files, use below command.
# find /tmp -type f -name ".*"
Part III – Search Files Based On Owners and Groups
22. Find Single File Based on User
To find all or single file called tecmint.txt under /root directory of owner root.
# find / -user root -name tecmint.txt
23. Find all Files Based on User
To find all files that belongs to user Tecmint under /home directory.
# find /home -user tecmint
24. Find all Files Based on Group
To find all files that belongs to group Developer under /home directory.
# find /home -group developer
25. Find Particular Files of User
To find all .txt files of user Tecmint under /home directory.
# find /home -user tecmint -iname "*.txt"
Part IV – Find Files and Directories Based on Date and Time
26. Find Last 50 Days Modified Files
To find all the files which are modified 50 days back.
# find / -mtime 50
27. Find Last 50 Days Accessed Files
To find all the files which are accessed 50 days back.
# find / -atime 50
28. Find Last 50-100 Days Modified Files
To find all the files which are modified more than 50 days back and less than 100 days.
# find / -mtime +50 –mtime -100
29. Find Changed Files in Last 1 Hour
To find all the files which are changed in last 1 hour.
# find / -cmin -60
30. Find Modified Files in Last 1 Hour
To find all the files which are modified in last 1 hour.
# find / -mmin -60
31. Find Accessed Files in Last 1 Hour
To find all the files which are accessed in last 1 hour.
# find / -amin -60
Part V – Find Files and Directories Based on Size
32. Find 50MB Files
To find all 50MB files, use.
# find / -size 50M
33. Find Size between 50MB – 100MB
To find all the files which are greater than 50MB and less than 100MB.
# find / -size +50M -size -100M
34. Find and Delete 100MB Files
To find all 100MB files and delete them using one single command.
# find / -size +100M -exec rm -rf {} \;
35. Find Specific Files and Delete
Find all .mp3 files with more than 10MB and delete them using one single command.
# find / -type f -name *.mp3 -size +10M -exec ls -l {} \;
That’s it, We are ending this post here, In our next article we will discuss more about other Linux commands in depth with practical examples. Let us know your opinions on this article using our comment section.

Set File And Directory Permissions using Find Command

This wll set all the directores to 755 and all the files to 644

find  -type d -print | xargs chmod 755

find  -type f -print | xargs chmod 644

Permissions on Linux

File permission symbols
 
If you run the command
ls -l
in your home directory, you will get a list of files that may include something like this
 
-rw-r--r--   1  bob  users   1892   Jul 10  18:30 linux_course_notes.txt
 
This basically says, interpreting this from RIGHT to LEFT that the file, linux_course_notes.txt was created at 6:30 PM on July 10 and is 1892 bytes large. It belongs to the group users (i.e, the people who use this computer). It belongs to bob in particular and it is one (1) file. Then come the file permission symbols.
 
Let's look at what these symbols mean:
 
 
The dashes - separate the permissions into three types
 
 
The first part refers to the owner's (bob's) permissions.
 
 
The dash - before the rw means that this is a normal file that contains any type of data. A directory, for example, would have a d instead of a dash.
 
 
The rw that follows means that bob can read and write to (modify) his own file. That's pretty logical. If you own it, you can do what you want with it.
 
 
The second part of the these symbols after the second dash, are the permissions for the group. Linux can establish different types of groups for file access. In a one home computer environment anyone who uses the computer can read this file but cannot write to (modify) it. This is a completely normal situation. You, as a user, may want to take away the rights of others to read your file. We'll cover how to do that later.
 
 
After the two dashes (two here because there is no write permissions for the group) come the overall user permissions. Anyone who might have access to the computer from inside or outside (in the case of a network) can read this file. Once again, we can take away the possibility of people reading this file if we so choose.
 
Let's take a look at some other examples. An interesting place to look at different kinds of file permissions is the /bin directory. Here we have the commands that anybody can use on the Linux system. Let's look at the command for gzip, a file compression utility for Linux.
 
 
-rwxr-xr-x   1 root     root        53468 May  1  1999 gzip
 
As we see here, there are some differences.
 
 
The program name, date, bytes are all standard. Even though this is obviously different information, the idea is the same as before.
 
The changes are in the owner and group. Root owns the file and it is in the group "root". Root is actually the only member of that group.
 
The file is an executable (program) so that's why the letter x is among the symbols.
 
 
This file can be executed by everybody: the owner (root), the group (root) and all others that have access to the computer
 
As we mentioned, the file is a program, so there is no need for anybody other than root to "write" to the file, so there is no w permissions for it for anybody but root.
 
If we look at a file in /sbin which are files that only root can use or execute, the permissions would look like this:
 
 
-rwxr--r--   1 root     root         1065 Jan 14  1999 cron
 
'cron' is a program on Linux systems that allows programs to be run automatically at certain times and under certain conditions. As we can see here, only root, the owner of the file, is allowed to use this program. There are no xpermissions for the rest of the users.
 
 
We hope you enjoyed this little walk-through of file permissions in Linux. Now that we know what we're looking for, we can talk about changing certain permissions.
 
 
 
chmod
 
chmod is a Linux command that will let you "set permissions" (aka, assign who can read/write/execute) on a file.
 
 
chmod permissions file
 
chmod permission1_permission2_permission3 file
 
When using chmod, you need to be aware that there are three types of Linux users that you are setting permissions for. Therefore, when setting permissions, you are assigning them for "yourself", "your group" and "everyone else" in the world. These users are technically know as:
 
Owner
Group
World
Therefore, when setting permissions on a file, you will want to assign all three levels of permissions, and not just one user.
 
 
Think of the chmod command actually having the following syntax...
 
 
chmod owner group world FileName
 
Now that you understand that you are setting permissions for THREE user levels, you just have to wrap your head around what permissions you are able to set!
 
 
There are three types of permissions that Linux allows for each file.
 
read
write
execute
Putting it all together:
 
So, in laymen terms, if you wanted a file to be readable by everyone, and writable by only you, you would write the chmod command with the following structure.
 
 
COMMAND : OWNER : GROUP : WORLD : PATH
 
chmod read & write read read FileName
chmod 6 4 4 myDoc.txt
 
Wait! What are those numbers?!?
 
Computers like numbers, not words. Sorry. You will have to deal with it. Take a look at the following output of `ls -l`
 
 
 
 
-rw-r--r-- 1 gcawood iqnection 382 Dec 19 6:49 myDoc.txt
 
You will need to convert the word read or write or execute into the numeric equivalent (octal) based on the table below.
 
4 – read (r)
2 – write (w)
1 – execute (x)
Practical Examples
 
 
chmod 400 mydoc.txt – read by owner
 
chmod 040 mydoc.txt – read by group
 
chmod 004 mydoc.txt – read by anybody (other)
 
chmod 200 mydoc.txt – write by owner
 
chmod 020 mydoc.txt – write by group
 
chmod 002 mydoc.txt – write by anybody
 
chmod 100 mydoc.txt – execute by owner
 
chmod 010 mydoc.txt – execute by group
 
chmod 001 mydoc.txt – execute by anybody
 
 
Wait! I don't get it... there aren't enough permissions to do what I want!
 
Good call. You need to add up the numbers to get other types of permissions...
 
 
So, try wrapping your head around this!!
 
7 = 4+2+1 (read/write/execute)
 
6 = 4+2 (read/write)
 
5 = 4+1 (read/execute)
 
4 = 4 (read)
 
3 = 2+1 (write/execute)
 
2 = 2 (write)
 
1 = 1 (execute)
 
 
chmod 666 mydoc.txt – read/write by anybody! (the devil loves this one!)
 
chmod 755 mydoc.txt – rwx for owner, rx for group and rx for the world
 
chmod 777 mydoc.txt – read, write, execute for all! (may not be the best plan in the world...)

Format a drive in bash

Linux refers to hard drives as either "hdx" or "sdx," where x is a letter, starting with a, which represents the order in which the drive was added to or detected by the computer. The "hd" prefix is used for IDE and PATA (formerly just ATA), and the "sd" prefix is used for SCSI, SATA and USB drives. Usually, a number is also put at the end of "hdx" or "sdx" to denote different partitions on the same physical drive, but for the purpose of formatting, you only need to know which letter the drive you want to format is.

4
The examples given in this how-to are for a computer with two IDE hard drives attached as a master and slave. In this, the drives are "hda" and "hdb." You will need to determine the letter of the drive you want to format for your own setup. We will be formatting the drive hdb. For all examples, replace "hdb" with whatever your drive designation is.

5
You can see all the drives attached to your system by typing the command "ls /dev/hd*" or "ls /dev/sd*", depending on which type (IDE, SATA and so forth) the drives are. On the example system, the result of this command looks like "/dev/hda /dev/hda1 /dev/hda2 /dev/hdb /dev/hdb1". Theoperating system is installed on hda, which has two partitions (hda1 and hda2), and there is one partition on hdb and hdb1.

Using fdisk
6
First, you will use the fdisk command to erase any old partitions on the drive and create a new one. Any changes you make using fdisk are only made permanent if you then issue the "w" command before quitting, so feel free to play around a little if you like. If at any time you find yourself stuck, you can quit the program without saving changes by holding the "Ctrl" key and pressing "c."

7
At the command prompt, type "fdisk /dev/hdb", replacing the "hdb" with the letters for your drive. Upon opening, fdisk may give you a couple of warnings, all of which can be ignored. It then gives you a prompt that looks like this: Command (m for help):

8
Enter "p" to see the partition table of the drive. The first line of output from the "p" command will also tell you the size of the drive. This is a good way to double-check that you are working with the correct drive.

9
If there are any partitions already on the drive, they will be listed as the last lines of the "p" command. On our example, this looks like "/dev/hdb1", followed by some information about the partition's size and filesystem.

10
To delete any existing partitions, press "d" and then "Enter." It will ask you which partition number you wish to delete. The number of the partition is the number that follows hdb, so on our example system, we enter 1. If there are multiple partitions, repeat the "d" command for each one. You can always view the partition table again with the "p" command.

11
Once you have deleted all existing partitions on the drive, you are ready to make a new one. Type "n" and hit "Enter." Then press "p" to create a primary partition. It asks you for a partition number; enter "1." Now you are asked which cylinder the partition should start at. The beginning of the drive is the default, so just hit "Enter." Then, you are asked for the last cylinder. The end of the drive is the default, so you can just press "Enter" again.

12
Now you are back at fdisk's command prompt. Use the "p" command to check the partition table. You should now see your new partition at the bottom of the output. In the example, it lists as "/dev/hdb1."

13
You now need to set the filesystem type for your new partition with the "t" command. You are asked for the Hex code of the filesystem you wish to use. We will use the standard Linux ext2 filesystem, which is "83." If you are doing something special and know of a particular filesystem that you need to use, you can press "L" to see all the codes, which are one or two characters made up of the numbers 0 to 9 and the letters a to f.

14
Now just issue the "w" command to write your new partition table and exit fdisk.



Read more: How to Format a Hard Drive in Linux | eHow.com http://www.ehow.com/how_1000631_hard-drive-linux.html#ixzz1teq8yBrz

File Permissions on Unix

Understanding file permissions on Unix: a brief tutorial

(For files on AFS fileservers, see below)

Every user on a Unix system has a unique username, and is a member of at least one group (the primary group for that user). This group information is held in the password file (/etc/passwd). A user can also be a member of one or more other groups. The auxiliary group information is held in the file /etc/group. Only the administrator can create new groups or add/delete group members (one of the shortcomings of the system).

Every directory and file on the system has an owner, and also an associated group. It also has a set of permission flags which specify separate read, write and execute permissions for the 'user' (owner), 'group', and 'other' (everyone else with an account on the computer) The 'ls' command shows the permissions and group associated with files when used with the -l option. On some systems (e.g. Coos), the '-g' option is also needed to see the group information.

An example of the output produced by 'ls -l' is shown below.

drwx------ 2 richard staff  2048 Jan  2 1997  private
drwxrws--- 2 richard staff  2048 Jan  2 1997  admin
-rw-rw---- 2 richard staff 12040 Aug 20 1996  admin/userinfo
drwxr-xr-x 3 richard user   2048 May 13 09:27 public
Understanding how to read this output is useful to all unix users, but especially people using group access permissions.

Field 1:   a set of ten permission flags. 
Field 2:   link count (don't worry about this)
Field 3:   owner of the file
Field 4:   associated group for the file
Field 5:   size in bytes
Field 6-8: date of last modification (format varies, but always 3 fields)
Field 9:   name of file (possibly with path, depending on how ls was called)

The permission flags are read as follows (left to right)

position	Meaning
1	directory flag, 'd' if a directory, '-' if a normal file, something else occasionally may appear here for special devices.
2,3,4	read, write, execute permission for User (Owner) of file
5,6,7	read, write, execute permission for Group
8,9,10	read, write, execute permission for Other
value	Meaning
-	in any position means that flag is not set
r	file is readable by owner, group or other
w	file is writeable. On a directory, write access means you can add or delete files
x	file is executable (only for programs and shell scripts - not useful for data files). Execute permission on a directory means you can list the files in that directory
s	in the place where 'x' would normally go is called the set-UID or set-groupID flag.
On an executable program with set-UID or set-groupID, that program runs with the effective permissions of its owner or group.

 

For a directory, the set-groupID flag means that all files created inside that directory will inherit the group of the directory. Without this flag, a file takes on the primary group of the user creating the file. This property is important to people trying to maintain a directory as group accessible. The subdirectories also inherit the set-groupID property.
The default file permissions (umask):

Each user has a default set of permissions which apply to all files created by that user, unless the software explicitly sets something else. This is often called the 'umask', after the command used to change it. It is either inherited from the login process, or set in the .cshrc or .login file which configures an individual account, or it can be run manually.

Typically the default configuration is equivalent to typing 'umask 22' which produces permissions of:

-rw-r--r-- for regular files, or
drwxr-xr-x for directories.
In other words, user has full access, everyone else (group and other) has read access to files, lookup access to directories.

When working with group-access files and directories, it is common to use 'umask 2' which produces permissions of:

-rw-rw-r-- for regular files, or
drwxrwxr-x for directories.
For private work, use 'umask 77' which produces permissions:

-rw------- for regular files, or
drwx------ for directories.
The logic behind the number given to umask is not intuitive.

The command to change the permission flags is "chmod". Only the owner of a file can change its permissions.

The command to change the group of a file is "chgrp". Only the owner of a file can change its group, and can only change it to a group of which he is a member.

See the online manual pages for details of these commands on any particular system (e.g. "man chmod").

Examples of typical useage are given below:

chmod g+w myfile
give group write permission to "myfile", leaving all other permission flags alone 
chmod g-rw myfile
remove read and write access to "myfile", leaving all other permission flags alone 
chmod g+rwxs mydir
give full group read/write access to directory "mydir", also setting the set-groupID flag so that directories created inside it inherit the group 
chmod u=rw,go= privatefile
explicitly give user read/write access, and revoke all group and other access, to file 'privatefile' 
chmod -R g+rw .
give group read write access to this directory, and everything inside of it (-R = recursive) 
chgrp -R medi .
change the ownership of this directory to group 'medi' and everything inside of it (-R = recursive). The person issuing this command must own all the files or it will fail.
WARNINGS:

Putting 'umask 2' into a startup file (.login or .cshrc) will make these settings apply to everything you do unless manually changed. This can lead to giving group access to files such as saved email in your home directory, which is generally not desireable.

Making a file group read/write without checking what its group is can lead to accidentally giving access to almost everyone on the system. Normally all users are members of some default group such as "users", as well as being members of specific project-oriented groups. Don't give group access to "users" when you intended some other group.

Remember that to read a file, you need execute access to the directory it is in AND read access to the file itself. To write a file, your need execute access to the directory AND write access to the file. To create new files or delete files, you need write access to the directory. You also need execute access to all parent directories back to the root. Group access will break if a parent directory is made completely private.

AFS Access Control Lists (ACLs)

Files on the central AFS fileservers all have the traditional Unix permissions as explained above, but they are also controlled by Access Control Lists(ACL) which take precedence. They provide access levels more flexible than the user/group/other attribute bits, but they work on the level of complete directories, not files. The command to set and list ACLs is fs. 
"fs" is a big ugly command that does lots of things related to AFS filesystems depending on the arguments you call it with.

For details see the man pages for: fs_setacl, fs_listacl, fs_cleanacl, fs_copyacl

For brief help, do (e.g.) "fs help setacl"

The default is to give the same permissions to a new directory as are on the parent directory. In practice, this is usually to give complete rights to the owner of the directory, and lookup rights to any other user (equivalent to execute attribute on a directory).

To render a directory private, the simplest command is:

fs setacl -d DIRNAME -clear -a MYNAME all

- replace DIRNAME with the appropriate directory name (or "." for the current directory, and MYNAME with your login name.

Check it with:

fs listacl DIRNAME

It should reply with:

Access list for DIRNAME is
Normal rights:
  USERNAME rlidwka
(see man fs_setacl for a description of the meaning of the flags "rlidwka")

To explicitly give public read/lookup access, use:

fs setacl -d DIRNAME -a system:anyuser read

This can be abbreviated to

fs sa DIRNAME system:anyuser read

If "fs" is not found, or the man pages are not found, your paths are not set up correctly. I recommend you run /usr/local/bin/mknewdotfiles to correct that.