Ubuntu Server Utilities
Linux DIG Command Examples for DNS Lookup
dig redhat.com - dns lookup dig redhat.com MX - mx records lookup dig redhat.com NS View ALL DNS Records Types Using dig -t ANY dig -t ANY redhat.com DNS Reverse Look-up Using dig -x dig -x 209.132.183.81 Use a Specific DNS server Using dig @dnsserver dig @ns1.redhat.com redhat.com
Extract Images from PDF Files With PDFimages
apt-get install poppler-utils pdfimages --help for options eg pdfimages -j filename.pdf prefixforallfiles if the image doesn't convert to jpg. then use the mogrify to convert them. mogrify is part of imagemagick mogrify -format jpg *.pcm // this will convert all *.pcm files to jpg
Imagemagic
convert rose.jpg rose.png convert rose.jpg -resize 64x64 thumb_rose.jpg Only shrink larger images convert rose.jpg -resize 64x64\> rose.jpg Percentage Resize convert rose.jpg -resize 50% rose.jpg Rotate convert rose.jpg -rotate 90 rose.jpg Rotate and overite mogrify -rotate 90 img.jpg Batch Convert from one format to another mogrify -format jpg *.png this will convert all png files to jpg
Using Rsync
rsync options source destination
Rsync Local Dir to Amazon AWS servers rsync azvr --delete --progress -e "ssh -i /fordertopemfile/pemfile.pem" localfoldertosync ubuntu@ipaddress:/var/tmp/ Synchronize Two Directories in a Local Server: To sync two directories in a local computer, use the following rsync -zvr command. $ rsync -azvr /var/opt/installation/inventory/ /root/temp building file list ... done sva.xml svB.xml . sent 26385 bytes received 1098 bytes 54966.00 bytes/sec total size is 44867 speedup is 1.63 $ In the above rsync example: -a archive mode (which preserves timestamp) -z is to enable compression -v verbose -r indicates recursive Synchronize Only One File: To copy only one file, specify the file name to rsync command, as shown below $ rsync -v /var/lib/rpm/Pubkeys /root/temp/ Pubkeys sent 42 bytes received 12380 bytes 3549.14 bytes/sec total size is 12288 speedup is 0.99
Rsync Over SSH
With rsync, we can use SSH (Secure Shell) for data transfer, using SSH protocol while transferring our data you can be ensured that your data is being transferred in a secured connection with encryption so that nobody can read your data while it is being transferred over the wire on the internet. Also when we use rsync we need to provide the user/root password to accomplish that particular task, so using SSH option will send your logins in an encrypted manner so that your password will be safe. === Copy a File from a Remote Server to a Local Server with SSH === To specify a protocol with rsync you need to give “-e” option with protocol name you want to use. Here in this example, We will be using “ssh” with “-e” option and perform data transfer. rsync -avzhe ssh root@192.168.0.100:/root/install.log /tmp/ root@192.168.0.100's password: receiving incremental file list install.log sent 30 bytes received 8.12K bytes 1.48K bytes/sec total size is 30.74K speedup is 3.77>/dd>
Copy a File from a Local Server to a Remote Server with SSH
rsync -avzhe ssh backup.tar root@192.168.0.100:/backups/ root@192.168.0.100's password: sending incremental file list backup.tar sent 14.71M bytes received 31 bytes 1.28M bytes/sec total size is 16.18M speedup is 1.10
Use of –include and –exclude Options
These two options allows us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred. Here in this example, rsync command will include those files and directory only which starts with ‘R’ and exclude all other files and directory. rsync -avze ssh --include 'R*' --exclude '*' root@192.168.0.101:/var/lib/rpm/ /root/rpm root@192.168.0.101's password: receiving incremental file list created directory /root/rpm
Vi Text Editor
Main command mode commands
i puts you in insert mode where you can enter text. ESC key takes you out of insert mode. w -> go to the beginning of the next word b -> go to the beginning of the previous word e -> go to the end of the next word n| -> go to the 'n' columns of the current row $ -> go to the end of the current row ^ -> go to the first character of the current row G -> go to the end of the file gg -> go to the beginning of the file Gn -> go to 'n' row of the file i -> insert mode (before the cursor) a -> insert mode (after the cursor) o -> insert mode (under the current row) x -> delete one character nx -> delete 'n' characters dd -> delete the whole row ndd -> delete 'n' rows yy -> copy the current row p -> paste the copied row Rmystring -> replace next characters with the string 'mystring' /string -> find the next string 'string' ?string -> find the string 'string' backwards n -> repeat the search again u -> undo the last command
Main row editing mode commands
:w -> write the file :w myfile -> write the file as 'myfile' :q -> quit the file :wq -> write and quit the file :q! -> quit without saving the file :help -> open the help file (:q to return editing the file) :help topic -> open the help file showing the subject 'topic' (:q to return editing the file) :rs/string1/string2/options -> replace string1 with string2 withing the 'r' range; options may be: g (replace all), c (ask before any replacement) :set number -> set rows numbering * to move down:press j * to move up:press k * to move to the left:press h * to move to the right:press l You will have to tap each key for the number of lines or characters you want the cursor to move. In this case, I want to move down seven lines and then to the right. So I would hit j seven times, and hold down the l key until I got to the word “friend.” Not every file you’re going to work in will be so small; some will be a couple hundred lines long. If you look at the last line of our file, you’ll see that the editor tells you how many lines there are in the file. If you want to jump down, you can type :(line number), where the line number is the actual numeral. So if I wanted to jump down to line 16 (the line beginning with “q”), I’d type: :16 hit Enter, and there I’d be.
Deleting Text
You can delete text one character at a time simply by pressing x. The character that gets deleted will be the one the cursor highlights. If that seems too tedious, you can type dw, and delete the text a word at a time. Note that vi counts the spaces in between words as words too, so I’d have to type dw nine times to delete the line. Or you can just wipe out the entire line of text. To do that, just type dd. Since this is the fastest way to do it, I’m going to type: dd and the line that my cursor sits in is gone. Replacing text type r (this tells the editor I want to replace the highlighted character). If I wanted to change the letter c to s, I’d type r, then s. If I wanted to change it to – well, you get the picture:Type r and then the replacing character.
Adding Text
So far, we’ve been working in edit mode. In this mode, we can move around the file with impunity, deleting or replacing existing text without worry. To add text, we have to switch to the insert mode.Fortunately, this isn’t hard:Just move the cursor to the spot in the document you want to edit, hit i, and begin typing. The first thing you should know is that vi takes the insert command rather literally. Wherever your cursor is when you hit i will be the first point at which you can begin typing. So if you’re looking at a sentence like this: w means to overwrite the last version and the cursor is on the v in ” overwrite,” you’re going to start inserting text where the v used to be. So if you try to type the words “saved file” you’ll end up with a sentence that looks like this: w means to osaved fileverwrite the last version The easiest way around it is to position the cursor where you want it before you start typing. So, if I wanted to insert the phrase “saved file” after the word last, I would hit Escape to make sure I’m out of insert mode, move the cursor to the space between “last” and “version,” hit i for insert, and type in a space and then ” saved file.” Note:You cannot move around using the h, j, k, and l keys, or edit anything, while you’re in insert mode! To snap out of insert mode and back into edit mode, hit the Escape key. Any time I’m in vi and forget which mode I’m in (it can happen), I hit Escape before I do anything else.
Undoing Mistakes
So let’s say you haven’t quite got the hang of inserting text and you’ve just typed something very wrong. Will you have to delete all the text and start over? Nope – if you catch your errors quickly (like, right after you make them), you can always undo them. To undo any inserted text, hit the Escape key, then type u. It will undo the last action you did. If you’re already in edit mode, just type u to undo your last action.
Webmin
Webmin is a web-based interface for system administration for Unix. Using any modern web browser, you can setup user accounts, Apache, DNS, file sharing and much more. Webmin removes the need to manually edit Unix configuration files like /etc/passwd, and lets you manage a system from the console or remotely http://www.webmin.com/ After Installation: https://localhost:10000/ username root with your current password
Log watch
Logwatch is a modular log analyser that runs every night and mails you the results. It can also be run from command line.The output is by service and you can limit the output to one particular service. The subscripts which are responsible for the output, mostly convert the raw log lines in structured format. Logwatch generally ignores the time component in the output, that means, you will know that the reported event was logged in the requested range of time, but you will have to go to the raw log files to get the exact details. Install logwatch using the following comamnd sudo apt-get install logwatch This will install all the required packages Configuring Logwatch First you need to make sure you server is able to send the mails outside you can do this using postfix with SMTP server configuration Now you need to edit logwatch.conf file sudo nano /usr/share/logwatch/default.conf/logwatch.conf Change the following information Output = mail Format = html MailTo = test@gmail.com Save and exit the file If you want to know more available options check logcheck manpage using the following command from your terminal man logcheck Now edit the 00logwatch file sudo nano /etc/cron.daily/00logwatch add the following line /usr/sbin/logwatch --mailto test@gmail.com Save and exit the file
Using Tar Linux Shell Command
First we'll start with 'tar'. This program assembles various files into one package, commonly called a "tarball". Let's say you have some files - notes that you've taken during this course.
You have:
notes_1.txt notes_2.txt notes_3.txt notes_4.txt notes_5.txt
and you've placed them in a directory called /linux_course. You want to back them up and keep them on a floppy, let's say. You would type the following command to package them in a tarball.
tar -cvf linux_notes.tar notes*.txt
First, you have tar, the name of the program. Then you have the options, c (--create) v (--verbose-show what files they are) (f--file -make a file - should always be the last option) Then you have the name of the file you want to create ( linux_notes.tar) and the files you want to backup (notes*.txt).
This presupposes that you may have other files in the directory that you don't want to include. If you want to include ALL files in a directory, just substitute notes*.txt for *.*.
back up whole directories along with their corresponding subdirectories. Enter in the directory, let's say /home/bob/ and issue the command:
tar -cvf bob_backup.tar *
With one asterisk, you will include directories and files without extensions (my_file as opposed to my_file.txt). Be prepared to get a fairly voluminous tarball.
This is the first step in the backup process. Now let's look at the second step; the compression of these files.
Using 'gzip'
As we mentioned, 'tar' just assembles the files together into only one file. There is no reduction in the size of these files (the tarball might even be bigger!) Now we would have to do one more thing in order to reduce this file into a more manageable size: use 'gzip'.
gzip is the preferred compression tool for Linux. To reduce the size of your tar file, you would issue the following command:
gzip your_tar_file.tar
and the tar file would be compressed. You can also compress a regular file using the same command, but gzip is used primarily with tarballs.
The result would be a file like this: your_tar_file.tar.gz
The two file extensions show us that the file is a tarball and it is compressed with the 'gzip' format.
Putting it all together
tar
tar has an option built into it to use 'gzip' to zip the file at the same time you make the tarball. If you add z to the options, and change the name of the file to create to a .gz extension, you have the whole shebang in one step. Our previous example would be modified to this:
tar -czvf bob_backup.tar.gz *
Remember f should always be the last option.
UnTar
Using 'tar' and 'gzip' sort of supposes that you're going to want to "untar" and "unzip" these files at one point or another.
The easiest way for doing this is to use 'tar' for the whole process. You would locate the zipped tarball in question and then ask yourself a question:
Did I make any changes to the files inside the tarball after I made it? If you did, then you've got an old tarball. If you untarred it in the same directory, you'd overwrite the existing ones. If you would like a copy of the old file, untar it in a different directory. If you don't want the old files, then you should make a new tarball. It's pretty standard backup practice.
When you've decided what you want to do, to proceed with the "untarring", issue this command:
tar -zxvpf my_tar_file.tar.gz
I've used my preferred options. I'll explain them:
-z - unzip the file first -x - extract the files from the tarball -v - "verbose" (i.e tar tells you what files it's extracting) -p - preserves dates, permissions of the original files -f - use the file in question (if you don't specify this, tar just sort of sits around doing nothing)
The files are extracted and your original tarball is preserved (my_tar_file.tar.gz).
You can also untar the file and then use gzip separately. Just leave the z option out of the previous example and type:
gzip -d my_tar_file.tar.gz or gunzip my_tar_file.tar.gz (gunzip runs gzip -d "automagically"!)
These commands are good if you've just zipped a regular file (not a tarball).