Friday, October 22, 2010

Ranikot Fort in Kirthar Range, Sindh | PakSnaps

Saturday, October 16, 2010

20 Linux System Monitoring Tools

20 Linux System Monitoring Tools Every SysAdmin Should Know

Need to monitor Linux server performance? Try these built-in command and a few add-on tools. Most Linux distributions are equipped with tons of monitoring. These tools provide metrics which can be used to get information about system activities. You can use these tools to find the possible causes of a performance problem. The commands discussed below are some of the most basic commands when it comes to system analysis and debugging server issues such as:

Finding out bottlenecks.
Disk (storage) bottlenecks.
CPU and memory bottlenecks.
Network bottlenecks.

#1: top - Process Activity Command

The top program provides a dynamic real-time view of a running system i.e. actual process activity. By default, it displays the most CPU-intensive tasks running on the server and updates the list every five seconds.

Fig.01: Linux top command

Commonly Used Hot Keys

The top command provides several useful hot keys:

Hot Key	Usage
t	Displays summary information off and on.
m	Displays memory information off and on.
A	Sorts the display by top consumers of various system resources. Useful for quick identification of performance-hungry tasks on a system.
f	Enters an interactive configuration screen for top. Helpful for setting up top for a specific task.
o	Enables you to interactively select the ordering within top.
r	Issues renice command.
k	Issues kill command.
z	Turn on or off color/mono

#2: vmstat - System Activity, Hardware and System Information

The command vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.
# vmstat 3
Sample Outputs:

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st  0  0      0 2540988 522188 5130400    0    0     2    32    4    2  4  1 96  0  0  1  0      0 2540988 522188 5130400    0    0     0   720 1199  665  1  0 99  0  0  0  0      0 2540956 522188 5130400    0    0     0     0 1151 1569  4  1 95  0  0  0  0      0 2540956 522188 5130500    0    0     0     6 1117  439  1  0 99  0  0  0  0      0 2540940 522188 5130512    0    0     0   536 1189  932  1  0 98  0  0  0  0      0 2538444 522188 5130588    0    0     0     0 1187 1417  4  1 96  0  0  0  0      0 2490060 522188 5130640    0    0     0    18 1253 1123  5  1 94  0  0

Display Memory Utilization Slabinfo

# vmstat -m

Get Information About Active / Inactive Memory Pages

# vmstat -a
=> Related: How do I find out Linux Resource utilization to detect system bottlenecks?

#3: w - Find Out Who Is Logged on And What They Are Doing

w command displays information about the users currently on the machine, and their processes.
# w username # w vivek
Sample Outputs:

 17:58:47 up 5 days, 20:28,  2 users,  load average: 0.36, 0.26, 0.24 USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT root     pts/0    10.1.3.145       14:55    5.00s  0.04s  0.02s vim /etc/resolv.conf root     pts/1    10.1.3.145       17:43    0.00s  0.03s  0.00s w

#4: uptime - Tell How Long The System Has Been Running

The uptime command can be used to see how long the server has been running. The current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes.
# uptime
Output:

 18:02:41 up 41 days, 23:42,  1 user,  load average: 0.00, 0.00, 0.00

1 can be considered as optimal load value. The load can change from system to system. For a single CPU system 1 - 3 and SMP systems 6-10 load value might be acceptable.

#5: ps - Displays The Processes

ps command will report a snapshot of the current processes. To select all processes use the -A or -e option:
# ps -A
Sample Outputs:

  PID TTY          TIME CMD     1 ?        00:00:02 init     2 ?        00:00:02 migration/0     3 ?        00:00:01 ksoftirqd/0     4 ?        00:00:00 watchdog/0     5 ?        00:00:00 migration/1     6 ?        00:00:15 ksoftirqd/1 .... .....  4881 ?        00:53:28 java  4885 tty1     00:00:00 mingetty  4886 tty2     00:00:00 mingetty  4887 tty3     00:00:00 mingetty  4888 tty4     00:00:00 mingetty  4891 tty5     00:00:00 mingetty  4892 tty6     00:00:00 mingetty  4893 ttyS1    00:00:00 agetty 12853 ?        00:00:00 cifsoplockd 12854 ?        00:00:00 cifsdnotifyd 14231 ?        00:10:34 lighttpd 14232 ?        00:00:00 php-cgi 54981 pts/0    00:00:00 vim 55465 ?        00:00:00 php-cgi 55546 ?        00:00:00 bind9-snmp-stat 55704 pts/1    00:00:00 ps

ps is just like top but provides more information.

Show Long Format Output

# ps -Al
To turn on extra full mode (it will show command line arguments passed to process):
# ps -AlF

To See Threads ( LWP and NLWP)

# ps -AlFH

To See Threads After Processes

# ps -AlLm

Print All Process On The Server

# ps ax # ps axu

Print A Process Tree

# ps -ejH # ps axjf # pstree

Print Security Information

# ps -eo euser,ruser,suser,fuser,f,comm,label # ps axZ # ps -eM

See Every Process Running As User Vivek

# ps -U vivek -u vivek u

Set Output In a User-Defined Format

# ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm # ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm # ps -eopid,tt,user,fname,tmout,f,wchan

Display Only The Process IDs of Lighttpd

# ps -C lighttpd -o pid=
OR
# pgrep lighttpd
OR
# pgrep -u vivek php-cgi

Display The Name of PID 55977

# ps -p 55977 -o comm=

Find Out The Top 10 Memory Consuming Process

# ps -auxf | sort -nr -k 4 | head -10

Find Out top 10 CPU Consuming Process

# ps -auxf | sort -nr -k 3 | head -10

#6: free - Memory Usage

The command free displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.
# free
Sample Output:

            total       used       free     shared    buffers     cached Mem:      12302896    9739664    2563232          0     523124    5154740 -/+ buffers/cache:    4061800    8241096 Swap:      1052248          0    1052248

=> Related: :

#7: iostat - Average CPU Load, Disk Activity

The command iostat report Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS).
# iostat
Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in) 	06/26/2009  avg-cpu:  %user   %nice %system %iowait  %steal   %idle            3.50    0.09    0.51    0.03    0.00   95.86  Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn sda              22.04        31.88       512.03   16193351  260102868 sda1              0.00         0.00         0.00       2166        180 sda2             22.04        31.87       512.03   16189010  260102688 sda3              0.00         0.00         0.00       1615          0

#8: sar - Collect and Report System Activity

The sar command is used to collect, report, and save system activity information. To see network counter, enter:
# sar -n DEV | more
To display the network counters from the 24th:
# sar -n DEV -f /var/log/sa/sa24 | more
You can also display real time usage using sar:
# sar 4 5
Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in) 		06/26/2009  06:45:12 PM       CPU     %user     %nice   %system   %iowait    %steal     %idle 06:45:16 PM       all      2.00      0.00      0.22      0.00      0.00     97.78 06:45:20 PM       all      2.07      0.00      0.38      0.03      0.00     97.52 06:45:24 PM       all      0.94      0.00      0.28      0.00      0.00     98.78 06:45:28 PM       all      1.56      0.00      0.22      0.00      0.00     98.22 06:45:32 PM       all      3.53      0.00      0.25      0.03      0.00     96.19 Average:          all      2.02      0.00      0.27      0.01      0.00     97.70

#9: mpstat - Multiprocessor Usage

The mpstat command displays activities for each available processor, processor 0 being the first one. mpstat -P ALL to display average CPU utilization per processor:
# mpstat -P ALL
Sample Output:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in)	 	06/26/2009  06:48:11 PM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s 06:48:11 PM  all    3.50    0.09    0.34    0.03    0.01    0.17    0.00   95.86   1218.04 06:48:11 PM    0    3.44    0.08    0.31    0.02    0.00    0.12    0.00   96.04   1000.31 06:48:11 PM    1    3.10    0.08    0.32    0.09    0.02    0.11    0.00   96.28     34.93 06:48:11 PM    2    4.16    0.11    0.36    0.02    0.00    0.11    0.00   95.25      0.00 06:48:11 PM    3    3.77    0.11    0.38    0.03    0.01    0.24    0.00   95.46     44.80 06:48:11 PM    4    2.96    0.07    0.29    0.04    0.02    0.10    0.00   96.52     25.91 06:48:11 PM    5    3.26    0.08    0.28    0.03    0.01    0.10    0.00   96.23     14.98 06:48:11 PM    6    4.00    0.10    0.34    0.01    0.00    0.13    0.00   95.42      3.75 06:48:11 PM    7    3.30    0.11    0.39    0.03    0.01    0.46    0.00   95.69     76.89

#10: pmap - Process Memory Usage

The command pmap report memory map of a process. Use this command to find out causes of memory bottlenecks.
# pmap -d PID
To display process memory information for pid # 47394, enter:
# pmap -d 47394
Sample Outputs:

47394:   /usr/bin/php-cgi Address           Kbytes Mode  Offset           Device    Mapping 0000000000400000    2584 r-x-- 0000000000000000 008:00002 php-cgi 0000000000886000     140 rw--- 0000000000286000 008:00002 php-cgi 00000000008a9000      52 rw--- 00000000008a9000 000:00000   [ anon ] 0000000000aa8000      76 rw--- 00000000002a8000 008:00002 php-cgi 000000000f678000    1980 rw--- 000000000f678000 000:00000   [ anon ] 000000314a600000     112 r-x-- 0000000000000000 008:00002 ld-2.5.so 000000314a81b000       4 r---- 000000000001b000 008:00002 ld-2.5.so 000000314a81c000       4 rw--- 000000000001c000 008:00002 ld-2.5.so 000000314aa00000    1328 r-x-- 0000000000000000 008:00002 libc-2.5.so 000000314ab4c000    2048 ----- 000000000014c000 008:00002 libc-2.5.so ..... ...... .. 00002af8d48fd000       4 rw--- 0000000000006000 008:00002 xsl.so 00002af8d490c000      40 r-x-- 0000000000000000 008:00002 libnss_files-2.5.so 00002af8d4916000    2044 ----- 000000000000a000 008:00002 libnss_files-2.5.so 00002af8d4b15000       4 r---- 0000000000009000 008:00002 libnss_files-2.5.so 00002af8d4b16000       4 rw--- 000000000000a000 008:00002 libnss_files-2.5.so 00002af8d4b17000  768000 rw-s- 0000000000000000 000:00009 zero (deleted) 00007fffc95fe000      84 rw--- 00007ffffffea000 000:00000   [ stack ] ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ] mapped: 933712K    writeable/private: 4304K    shared: 768000K

The last line is very important:

mapped: 933712K total amount of memory mapped to files
writeable/private: 4304K the amount of private address space
shared: 768000K the amount of address space this process is sharing with others

#11 and #12: netstat and ss - Network Statistics

The command netstat displays network connections, routing tables, interface statistics, masquerade connections, and multicast memberships. ss command is used to dump socket statistics. It allows showing information similar to netstat. See the following resources about ss and netstat commands:

#13: iptraf - Real-time Network Statistics

The iptraf command is interactive colorful IP LAN monitor. It is an ncurses-based IP LAN monitor that generates various network statistics including TCP info, UDP counts, ICMP and OSPF information, Ethernet load info, node stats, IP checksum errors, and others. It can provide the following info in easy to read format:

Network traffic statistics by TCP connection
IP traffic statistics by network interface
Network traffic statistics by protocol
Network traffic statistics by TCP/UDP port and by packet size
Network traffic statistics by Layer2 address

Fig.02: General interface statistics: IP traffic statistics by network interface

Fig.03 Network traffic statistics by TCP connection

#14: tcpdump - Detailed Network Traffic Analysis

The tcpdump is simple command that dump traffic on a network. However, you need good understanding of TCP/IP protocol to utilize this tool. For.e.g to display traffic info about DNS, enter:
# tcpdump -i eth1 'udp port 53'
To display all IPv4 HTTP packets to and from port 80, i.e. print only packets that contain data, not, for example, SYN and FIN packets and ACK-only packets, enter:
# tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2))>>2)) != 0)'
To display all FTP session to 202.54.1.5, enter:
# tcpdump -i eth1 'dst 202.54.1.5 and (port 21 or 20'
To display all HTTP session to 192.168.1.5:
# tcpdump -ni eth0 'dst 192.168.1.5 and tcp and port http'
Use wireshark to view detailed information about files, enter:
# tcpdump -n -i eth1 -s 0 -w output.txt src or dst port 80

#15: strace - System Calls

Trace system calls and signals. This is useful for debugging webserver and other server problems. See how to use to trace the process and see What it is doing.

#16: /Proc file system - Various Kernel Statistics

/proc file system provides detailed information about various hardware devices and other Linux kernel information. See Linux kernel /proc documentations for further details. Common /proc examples:
# cat /proc/cpuinfo # cat /proc/meminfo # cat /proc/zoneinfo # cat /proc/mounts

17#: Nagios - Server And Network Monitoring

Nagios is a popular open source computer system and network monitoring application software. You can easily monitor all your hosts, network equipment and services. It can send alert when things go wrong and again when they get better. FAN is "Fully Automated Nagios". FAN goals are to provide a Nagios installation including most tools provided by the Nagios Community. FAN provides a CDRom image in the standard ISO format, making it easy to easilly install a Nagios server. Added to this, a wide bunch of tools are including to the distribution, in order to improve the user experience around Nagios.

18#: Cacti - Web-based Monitoring Tool

Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices. It can provide data about network, CPU, memory, logged in users, Apache, DNS servers and much more. See how to install and configure Cacti network graphing tool under CentOS / RHEL.

#19: KDE System Guard - Real-time Systems Reporting and Graphing

KSysguard is a network enabled task and system monitor application for KDE desktop. This tool can be run over ssh session. It provides lots of features such as a client/server architecture that enables monitoring of local and remote hosts. The graphical front end uses so-called sensors to retrieve the information it displays. A sensor can return simple values or more complex information like tables. For each type of information, one or more displays are provided. Displays are organized in worksheets that can be saved and loaded independently from each other. So, KSysguard is not only a simple task manager but also a very powerful tool to control large server farms.

Fig.05 KDE System Guard {Image credit: Wikipedia}

See the KSysguard handbook for detailed usage.

#20: Gnome System Monitor - Real-time Systems Reporting and Graphing

The System Monitor application enables you to display basic system information and monitor system processes, usage of system resources, and file systems. You can also use System Monitor to modify the behavior of your system. Although not as powerful as the KDE System Guard, it provides the basic information which may be useful for new users:

Displays various basic information about the computer's hardware and software.
Linux Kernel version
GNOME version
Hardware
Installed memory
Processors and speeds
System Status
Currently available disk space
Processes
Memory and swap space
Network usage
File Systems
Lists all mounted filesystems along with basic information about each.

Fig.06 The Gnome System Monitor application

Bonus: Additional Tools

A few more tools:

nmap - scan your server for open ports.
lsof - list open files, network connections and much more.
ntop web based tool - ntop is the best tool to see network usage in a way similar to what top command does for processes i.e. it is network traffic monitoring software. You can see network status, protocol wise distribution of traffic for UDP, TCP, DNS, HTTP and other protocols.
Conky - Another good monitoring tool for the X Window System. It is highly configurable and is able to monitor many system variables including the status of the CPU, memory, swap space, disk storage, temperatures, processes, network interfaces, battery power, system messages, e-mail inboxes etc.
GKrellM - It can be used to monitor the status of CPUs, main memory, hard disks, network interfaces, local and remote mailboxes, and many other things.
vnstat - vnStat is a console-based network traffic monitor. It keeps a log of hourly, daily and monthly network traffic for the selected interface(s).
htop - htop is an enhanced version of top, the interactive process viewer, which can display the list of processes in a tree form.
mtr - mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.

Linux: Setup a transparent proxy with Squid in three easy steps

by LinuxTitli · 236 comments

Y'day I got a chance to play with Squid and iptables. My job was simple : Setup Squid proxy as a transparent server.

Main benefit of setting transparent proxy is you do not have to setup up individual browsers to work with proxies.

My Setup:

i) System: HP dual Xeon CPU system with 8 GB RAM (good for squid).
ii) Eth0: IP:192.168.1.1
iii) Eth1: IP: 192.168.2.1 (192.168.2.0/24 network (around 150 windows XP systems))
iv) OS: Red Hat Enterprise Linux 4.0 (Following instruction should work with Debian and all other Linux distros)

Eth0 connected to internet and eth1 connected to local lan i.e. system act as router.

Server Configuration

Step #1 : Squid configuration so that it will act as a transparent proxy
Step #2 : Iptables configuration

a) Configure system as router
b) Forward all http requests to 3128 (DNAT)

Step #3: Run scripts and start squid service

First, Squid server installed (use up2date squid) and configured by adding following directives to file:
# vi /etc/squid/squid.conf

Modify or add following squid directives:
httpd_accel_host virtual httpd_accel_port 80 httpd_accel_with_proxy on httpd_accel_uses_host_header on acl lan src 192.168.1.1 192.168.2.0/24 http_access allow localhost http_access allow lan

Where,

httpd_accel_host virtual: Squid as an httpd accelerator
httpd_accel_port 80: 80 is port you want to act as a proxy
httpd_accel_with_proxy on: Squid act as both a local httpd accelerator and as a proxy.
httpd_accel_uses_host_header on: Header is turned on which is the hostname from the URL.
acl lan src 192.168.1.1 192.168.2.0/24: Access control list, only allow LAN computers to use squid
http_access allow localhost: Squid access to LAN and localhost ACL only
http_access allow lan: -- same as above --

Here is the complete listing of squid.conf for your reference (grep will remove all comments and sed will remove all empty lines, thanks to David Klein for quick hint ):
# grep -v "^#" /etc/squid/squid.conf | sed -e '/^$/d'

OR, try out sed (thanks to kotnik for small sed trick)
# cat /etc/squid/squid.conf | sed '/ *#/d; /^ *$/d'

Output:
hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? no_cache deny QUERY hosts_file /etc/hosts refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern . 0 20% 4320 acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl purge method PURGE acl CONNECT method CONNECT cache_mem 1024 MB http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !Safe_ports http_access deny CONNECT !SSL_ports acl lan src 192.168.1.1 192.168.2.0/24 http_access allow localhost http_access allow lan http_access deny all http_reply_access allow all icp_access allow all visible_hostname myclient.hostname.com httpd_accel_host virtual httpd_accel_port 80 httpd_accel_with_proxy on httpd_accel_uses_host_header on coredump_dir /var/spool/squid

Iptables configuration

Next, I had added following rules to forward all http requests (coming to port 80) to the Squid server port 3128 :
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j DNAT --to 192.168.1.1:3128 iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

Here is complete shell script. Script first configure Linux system as router and forwards all http request to port 3128 (Download the fw.proxy shell script):
#!/bin/sh # squid server IP SQUID_SERVER="192.168.1.1" # Interface connected to Internet INTERNET="eth0" # Interface connected to LAN LAN_IN="eth1" # Squid port SQUID_PORT="3128" # DO NOT MODIFY BELOW # Clean old firewall iptables -F iptables -X iptables -t nat -F iptables -t nat -X iptables -t mangle -F iptables -t mangle -X # Load IPTABLES modules for NAT and IP conntrack support modprobe ip_conntrack modprobe ip_conntrack_ftp # For win xp ftp client #modprobe ip_nat_ftp echo 1 > /proc/sys/net/ipv4/ip_forward # Setting default filter policy iptables -P INPUT DROP iptables -P OUTPUT ACCEPT # Unlimited access to loop back iptables -A INPUT -i lo -j ACCEPT iptables -A OUTPUT -o lo -j ACCEPT # Allow UDP, DNS and Passive FTP iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT # set this system as a router for Rest of LAN iptables --table nat --append POSTROUTING --out-interface $INTERNET -j MASQUERADE iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT # unlimited access to LAN iptables -A INPUT -i $LAN_IN -j ACCEPT iptables -A OUTPUT -o $LAN_IN -j ACCEPT # DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT # if it is same system iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT # DROP everything and Log it iptables -A INPUT -j LOG iptables -A INPUT -j DROP

Save shell script. Execute script so that system will act as a router and forward the ports:
# chmod +x /etc/fw.proxy # /etc/fw.proxy # service iptables save # chkconfig iptables on

Start or Restart the squid:
# /etc/init.d/squid restart # chkconfig squid on

Desktop / Client computer configuration

Point all desktop clients to your eth1 IP address (192.168.2.1) as Router/Gateway (use DHCP to distribute this information). You do not have to setup up individual browsers to work with proxies.

How do I test my squid proxy is working correctly?

See access log file /var/log/squid/access.log:
# tail -f /var/log/squid/access.log

Above command will monitor all incoming request and log them to /var/log/squid/access_log file. Now if somebody accessing a website through browser, squid will log information.

Problems and solutions

(a) Windows XP FTP Client

All Desktop client FTP session request ended with an error:
Illegal PORT command.

I had loaded the ip_nat_ftp kernel module. Just type the following command press Enter and voila!
# modprobe ip_nat_ftp

Please note that modprobe command is already added to a shell script (above).

(b) Port 443 redirection

I had block out all connection request from our router settings except for our proxy (192.168.1.1) server. So all ports including 443 (https/ssl) request denied. You cannot redirect port 443, from debian mailing list, "Long answer: SSL is specifically designed to prevent "man in the middle" attacks, and setting up squid in such a way would be the same as such a "man in the middle" attack. You might be able to successfully achive this, but not without breaking the encryption and certification that is the point behind SSL".

Therefore, I had quickly reopen port 443 (router firewall) for all my LAN computers and problem was solved.

(c) Squid Proxy authentication in a transparent mode

You cannot use Squid authentication with a transparently intercepting proxy.

Samba-HOWTO-Collection on Red Hat Linux

# This is the main Samba configuration file. You should read the
# smb.conf(5) manual page in order to understand the options listed
# here. Samba has a huge number of configurable options (perhaps too
# many!) most of which are not shown in this example
#
# For a step to step guide on installing, configuring and using samba,
# read the Samba-HOWTO-Collection. This may be obtained from:
# http://www.samba.org/samba/docs/Samba-HOWTO-Collection.pdf
#
# Many working examples of smb.conf files can be found in the
# Samba-Guide which is generated daily and can be downloaded from:
# http://www.samba.org/samba/docs/Samba-Guide.pdf
#
# Any line which starts with a ; (semi-colon) or a # (hash)
# is a comment and is ignored. In this example we will use a #
# for commentry and a ; for parts of the config file that you
# may wish to enable
#
# NOTE: Whenever you modify this file you should run the command "testparm"
# to check that you have not made any basic syntactic errors.
#
#---------------
# SELINUX NOTES:
#
# If you want to use the useradd/groupadd family of binaries please run:
# setsebool -P samba_domain_controller on
#
# If you want to share home directories via samba please run:
# setsebool -P samba_enable_home_dirs on
#
# If you create a new directory you want to share you should mark it as
# "samba-share_t" so that selinux will let you write into it.
# Make sure not to do that on system directories as they may already have
# been marked with othe SELinux labels.
#
# Use ls -ldZ /path to see which context a directory has
#
# Set labels only on directories you created!
# To set a label use the following: chcon -t samba_share_t /path
#
# If you need to share a system created directory you can use one of the
# following (read-only/read-write):
# setsebool -P samba_export_all_ro on
# or
# setsebool -P samba_export_all_rw on
#
# If you want to run scripts (preexec/root prexec/print command/...) please
# put them into the /var/lib/samba/scripts directory so that smbd will be
# allowed to run them.
# Make sure you COPY them and not MOVE them so that the right SELinux context
# is applied, to check all is ok use restorecon -R -v /var/lib/samba/scripts
#
#--------------
#
#======================= Global Settings =====================================

[global]
#--authconfig--start-line--

# Generated by authconfig on 2010/09/02 18:04:52
# DO NOT EDIT THIS SECTION (delimited by --start-line--/--end-line--)
# Any modification may be deleted or altered by authconfig in future

workgroup = workgroup
# password server = hamdardisb
#security = user
#idmap uid = 16777216-33554431
#idmap gid = 16777216-33554431
#template shell = /bin/false
#winbind use default domain = false
#winbind offline logon = false
netbios name=installer
security = share

#--authconfig--end-line--

# ----------------------- Network Related Options -------------------------
#
# workgroup = NT-Domain-Name or Workgroup-Name, eg: MIDEARTH
#
# server string is the equivalent of the NT Description field
#
# netbios name can be used to specify a server name not tied to the hostname
#
# Interfaces lets you configure Samba to use multiple interfaces
# If you have multiple network interfaces then you can list the ones
# you want to listen on (never omit localhost)
#
# Hosts Allow/Hosts Deny lets you restrict who can connect, and you can
# specifiy it as a per share option as well
#
; workgroup = MYGROUP
server string = Samba Server Version %v

; netbios name = MYSERVER

; interfaces = lo eth0 192.168.12.2/24 192.168.13.2/24
; hosts allow = 127. 192.168.12. 192.168.13.

# --------------------------- Logging Options -----------------------------
#
# Log File let you specify where to put logs and how to split them up.
#
# Max Log Size let you specify the max size log files should reach

# logs split per machine
; log file = /var/log/samba/%m.log
# max 50KB per log file, then rotate
; max log size = 50

# ----------------------- Standalone Server Options ------------------------
#
# Security can be set to user, share(deprecated) or server(deprecated)
#
# Backend to store user information in. New installations should
# use either tdbsam or ldapsam. smbpasswd is available for backwards
# compatibility. tdbsam requires no further configuration.

; security = share
passdb backend = tdbsam

# ----------------------- Domain Members Options ------------------------
#
# Security must be set to domain or ads
#
# Use the realm option only with security = ads
# Specifies the Active Directory realm the host is part of
#
# Backend to store user information in. New installations should
# use either tdbsam or ldapsam. smbpasswd is available for backwards
# compatibility. tdbsam requires no further configuration.
#
# Use password server option only with security = server or if you can't
# use the DNS to locate Domain Controllers
# The argument list may include:
# password server = My_PDC_Name [My_BDC_Name] [My_Next_BDC_Name]
# or to auto-locate the domain controller/s
# password server = *

; security = domain
; passdb backend = tdbsam
; realm = MY_REALM

; password server =

# ----------------------- Domain Controller Options ------------------------
#
# Security must be set to user for domain controllers
#
# Backend to store user information in. New installations should
# use either tdbsam or ldapsam. smbpasswd is available for backwards
# compatibility. tdbsam requires no further configuration.
#
# Domain Master specifies Samba to be the Domain Master Browser. This
# allows Samba to collate browse lists between subnets. Don't use this
# if you already have a Windows NT domain controller doing this job
#
# Domain Logons let Samba be a domain logon server for Windows workstations.
#
# Logon Scrpit let yuou specify a script to be run at login time on the client
# You need to provide it in a share called NETLOGON
#
# Logon Path let you specify where user profiles are stored (UNC path)
#
# Various scripts can be used on a domain controller or stand-alone
# machine to add or delete corresponding unix accounts
#
; security = user
; passdb backend = tdbsam

; domain master = yes
; domain logons = yes

# the login script name depends on the machine name
; logon script = %m.bat
# the login script name depends on the unix user used
; logon script = %u.bat
; logon path = \\%L\Profiles\%u
# disables profiles support by specifing an empty path
; logon path =

; add user script = /usr/sbin/useradd "%u" -n -g users
; add group script = /usr/sbin/groupadd "%g"
; add machine script = /usr/sbin/useradd -n -c "Workstation (%u)" -M -d /nohome -s /bin/false "%u"
; delete user script = /usr/sbin/userdel "%u"
; delete user from group script = /usr/sbin/userdel "%u" "%g"
; delete group script = /usr/sbin/groupdel "%g"

# ----------------------- Browser Control Options ----------------------------
#
# set local master to no if you don't want Samba to become a master
# browser on your network. Otherwise the normal election rules apply
#
# OS Level determines the precedence of this server in master browser
# elections. The default value should be reasonable
#
# Preferred Master causes Samba to force a local browser election on startup
# and gives it a slightly higher chance of winning the election
; local master = no
; os level = 33
; preferred master = yes

#----------------------------- Name Resolution -------------------------------
# Windows Internet Name Serving Support Section:
# Note: Samba can be either a WINS Server, or a WINS Client, but NOT both
#
# - WINS Support: Tells the NMBD component of Samba to enable it's WINS Server
#
# - WINS Server: Tells the NMBD components of Samba to be a WINS Client
#
# - WINS Proxy: Tells Samba to answer name resolution queries on
# behalf of a non WINS capable client, for this to work there must be
# at least one WINS Server on the network. The default is NO.
#
# DNS Proxy - tells Samba whether or not to try to resolve NetBIOS names
# via DNS nslookups.

; wins support = yes
; wins server = w.x.y.z
; wins proxy = no

; dns proxy = no

# --------------------------- Printing Options -----------------------------
#
# Load Printers let you load automatically the list of printers rather
# than setting them up individually
#
# Cups Options let you pass the cups libs custom options, setting it to raw
# for example will let you use drivers on your Windows clients
#
# Printcap Name let you specify an alternative printcap file
#
# You can choose a non default printing system using the Printing option

load printers = yes
cups options = raw

; printcap name = /etc/printcap
#obtain list of printers automatically on SystemV
; printcap name = lpstat
; printing = cups

# --------------------------- Filesystem Options ---------------------------
#
# The following options can be uncommented if the filesystem supports
# Extended Attributes and they are enabled (usually by the mount option
# user_xattr). Thess options will let the admin store the DOS attributes
# in an EA and make samba not mess with the permission bits.
#
# Note: these options can also be set just per share, setting them in global
# makes them the default for all shares

; map archive = no
; map hidden = no
; map read only = no
; map system = no
; store dos attributes = yes

#============================ Share Definitions ==============================

[homes]
comment = Home Directories
browseable = no
writable = yes
; valid users = %S
; valid users = MYDOMAIN\%S

[printers]
comment = All Printers
path = /var/spool/samba
browseable = no
vss ok = yes
writable = no
printable = yes

# Un-comment the following and create the netlogon directory for Domain Logons
; [netlogon]
; comment = Network Logon Service
; path = /var/lib/samba/netlogon
; guest ok = yes
; writable = no
; share modes = no

# Un-comment the following to provide a specific roving profile share
# the default is to use the user's home directory
; [Profiles]
; path = /var/lib/samba/profiles
; browseable = no
; vss ok = yes

[vss]
comment = control your source
path = /usr/local/vss
available = yes
valid users = vss
read only = no
browseable = yes
writable = yes
#force user = nobody
#force group = nobody
#read only = No
#guest ok = Yes

# A publicly accessible directory, but read only, except for people in
# the "staff" group
; [public]
; comment = Public Stuff
; path = /home/samba
; public = yes
; writable = yes
; printable = no
; write list = +staff

Thursday, October 14, 2010

Adding New Drives to Linux Machine

Adding New Drives

There are many reasons why you would need to add a new drive to your Linux box. You might have out-grown your current space limitations, or you may want to add a separate drive for a specific project or service. In any case, if you follow this guide, you should have no problems. First, you must be familiar with the naming scheme Linux uses for your drives. For purposes if this article, everything highlighted in blue is the command you must type at the terminal prompt. Anything hightlighted in red indicates changes made by a previous command or something you should pay special attention to. Anything highlighted in green is simply the standard output from a terminal or issued command. The machine used for this guide is running Red Hat Entperise Linux.

Conventional Naming
This article assumes you know the difference between primary, extended and logical partitions. In the following example, I added a SCSI hard drive with one primary partition. With that being said, I'll continue explaining the naming scheme. Linux gives each drive a 3 letter name followed by a partition number. If you are using IDE drives, these will all be named hd**. Where hd denotes the drive and the next two variables are aassociated with IDE order (primary master, primary slave, etc) and partition number. So, your primary master drive will always be named hda. If you are using SCSI drives, everything remains the same except instead of using hd, drives are named sd**. Your SCSI primary master drive will always be named sda. All of these devices reside under the /dev portion of your Linux file system. For instance, as root, you can issue the following command to view the naming combinations possible for your system. Remember, in this example, I am using SCSI hardware. If you are using IDE, simply use hd is place of sd here.

 [root@roswell root]# ls /dev/sda* /dev/sda     /dev/sdab10  /dev/sdad12  /dev/sdaf14  /dev/sdah2   /dev/sdaj4   /dev/sdal6 /dev/sda1    /dev/sdab11  /dev/sdad13  /dev/sdaf15  /dev/sdah3   /dev/sdaj5   /dev/sdal7 /dev/sda10   /dev/sdab12  /dev/sdad14  /dev/sdaf2   /dev/sdah4   /dev/sdaj6   /dev/sdal8 /dev/sda11   /dev/sdab13  /dev/sdad15  /dev/sdaf3   /dev/sdah5   /dev/sdaj7   /dev/sdal9 /dev/sda12   /dev/sdab14  /dev/sdad2   /dev/sdaf4   /dev/sdah6   /dev/sdaj8   /dev/sdam /dev/sda13   /dev/sdab15  /dev/sdad3   /dev/sdaf5   /dev/sdah7   /dev/sdaj9   /dev/sdam1 /dev/sda14   /dev/sdab2   /dev/sdad4   /dev/sdaf6   /dev/sdah8   /dev/sdak    /dev/sdam10 ---- Some results omitted** ---- [root@roswell root]#

Note that I omitted a large portion of the results because they were huge! Remember that this list is not dependent on the current number of hard drives in the system, but the maximum naming possibilities for any number of drives and partitions. Do not be surprised if your results are rather lengthy as well.

Creating, Mounting, and Configuration New Partitions

Before adding an extra drive, this machine had 2 physical drives. Both of them were named accordingly (sda and sdb) before the new drive was added. The second drive containing the swap partitions was automatically renamed when the new drive was added. Notice the command and output below:

 [root@roswell root]# df -h Filesystem            Size  Used Avail Use% Mounted on /dev/sda1             8.3G  2.4G  5.5G  30% / /dev/sda2              99M   26M   69M  27% /boot /dev/sdc1              16G   13G  2.3G  85% /export  <-- old sdb renamed to sdc by the Linux none                  250M     0  250M   0% /dev/shm [root@roswell root]#

This command simply lists all currently mounted drives, their names, and space usage. Notice that sdb is not presently mounted. However, we know that it exists otherwise, there would not be an sdc present. I could not add my new drive as sdc because my SCSI hotswap drive cage reserves the first two slots for 1.5" drives. So I was forced to make the new drive sdb because it is a 1.5" drive.

Setting Partitions

You should be fairly familiar with fdisk. The commands are somewhat different than it's DOS equivalent. See the following commands and output:

 [root@roswell root]# fdisk /dev/sdb  Command (m for help): m Command action    a   toggle a bootable flag    b   edit bsd disklabel    c   toggle the dos compatibility flag    d   delete a partition    l   list known partition types    m   print this menu    n   add a new partition    o   create a new empty DOS partition table    p   print the partition table    q   quit without saving changes    s   create a new empty Sun disklabel    t   change a partition's system id    u   change display/entry units    v   verify the partition table    w   write table to disk and exit    x   extra functionality (experts only)  Command (m for help):

If there is a problem, and there is no drive associated with /dev/sdb, you will get an error message. Remember, that nothing will actually be executed until you issue a w command. It's always a good idea to read through the variables of your commands. Doing so will ensure that you aren't forgetting anything. Let's get started!

 Command (m for help): p  Disk /dev/sdb: 50.0 GB, 50019202560 bytes 255 heads, 63 sectors/track, 6081 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes     Device Boot    Start       End    Blocks   Id  System  Command (m for help):

If you issue a p command, you will see any partitions that currently exist on the drive. You can see by the output above there are no existing partitions. This drive is un-partitionedd and unformatted. To create a new partition, is the n command.

 Command (m for help): n Command action    e   extended    p   primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-6081, default 1): 1 Last cylinder or +size or +sizeM or +sizeK (1-6081, default 6081): 6081  Command (m for help):

In the output above notice that interval I selected for the cylinders. Using the entire range allows you create one partition across the entire drive. So, in order to create a primary partition on /dev/sdb/ we issued the following commands:

We can check the partition specifications we just entered by using the p command again.

 Command (m for help): p  Disk /dev/sdb: 50.0 GB, 50019202560 bytes 255 heads, 63 sectors/track, 6081 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes     Device Boot    Start       End    Blocks   Id  System /dev/sdb1             1      6081  48845601   83  Linux  Command (m for help):

Notice the new partition (highlighted in red). However, we must issue a w command to finalize it. If you messed anything up, you can use the d command and specify which partition you want to delete.

 Command (m for help): w The partition table has been altered!  Calling ioctl() to re-read partition table. Syncing disks. [root@roswell root]#

Formatting

Now that the partition has been created, you need to format the drive. You can format it with almost any file system you wish. However, the most common Linux formats are ext2 and ext3. Ext3 is simply a candy coated version of ext2 that adds a logging feature. You must specify which partition to format by calling the device and partition number like this:

 [root@roswell root]# mkfs -t ext3 /dev/sdb1 mke2fs 1.32 (09-Nov-2002) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 6111232 inodes, 12211400 blocks 610570 blocks (5.00%) reserved for the super user First data block=0 373 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks:          32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,          4096000, 7962624, 11239424  Writing inode tables: done                             Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done  This filesystem will be automatically checked every 38 mounts or 180 days, whichever comes first.  Use tune2fs -c or -i to override. [root@roswell root]#

What did we do there? Using the mkfs (make file system) command, we specified the type (using the -t) ext3 using the device and partition name (/dev/sdb1). You have successfull partitioned and formatted your new drive. Wait, you're not done yet. You will want to mount this partition to make it usable. You will also want this partition to mount automatically when you reboot the machine.

Mounting

In order to automatically mount a partition, you must edit the /etc/fstab file. The fstab file tells Linux where to mount all partitions located within the system. The output below shows the current fstab file before including the newly added drive:

 [root@roswell root]# vi /etc/fstab LABEL=/                 /                       ext3    defaults        1 1 LABEL=/boot             /boot                   ext3    defaults        1 2 none                    /dev/pts                devpts  gid=5,mode=620  0 0 LABEL=/export           /export                 ext3    defaults        1 2 none                    /proc                   proc    defaults        0 0 none                    /dev/shm                tmpfs   defaults        0 0 /dev/sdb2               swap                    swap    defaults        0 0 /dev/cdrom              /mnt/cdrom              udf,iso9660 noauto,owner,kudzu,ro 0 0 /dev/fd0                /mnt/floppy             auto    noauto,owner,kudzu 0 0

You may notice I viewed this file using vi. Vi is a simple text editor that may or may not be loaded on your Linux system. It is somewhat similar to emacs. In any case, both programs can perform the same task. We will mount the new partition as /media. Remember to create a directory named media, otherwise fstab won't be able to mount the partition. It is shown high-lighted red in the output below:

 LABEL=/                 /                       ext3    defaults        1 1 LABEL=/boot             /boot                   ext3    defaults        1 2 none                    /dev/pts                devpts  gid=5,mode=620  0 0 LABEL=/export           /export                 ext3    defaults        1 2 none                    /proc                   proc    defaults        0 0 none                    /dev/shm                tmpfs   defaults        0 0 /dev/sdb1               /media                  ext3    defaults        1 2 /dev/sdb2               swap                    swap    defaults        0 0 /dev/cdrom              /mnt/cdrom              udf,iso9660 noauto,owner,kudzu,ro 0 0 /dev/fd0                /mnt/floppy             auto    noauto,owner,kudzu 0 0

Next, issue a simple mount command providing the partition name:

 [root@roswell export]# mount /dev/sdb1 [root@roswell export]#

You're all done! You will be able to access the /media folder immediately and after the machine reboots as fstab will automatically re-mount it for you. If you want to verify the partition is successfully present and mounted, use the following commands:

 [root@roswell media]# mount /dev/sda1 on / type ext3 (rw) none on /proc type proc (rw) none on /dev/pts type devpts (rw,gid=5,mode=620) usbdevfs on /proc/bus/usb type usbdevfs (rw) /dev/sda2 on /boot type ext3 (rw) /dev/sdc1 on /export type ext3 (rw) none on /dev/shm type tmpfs (rw) /dev/sdb1 on /media type ext3 (rw) [root@roswell media]#

The red line shows our new drive freshly mounted. You can check the space usage if you issue the following command.

 [root@roswell media]# df -h Filesystem            Size  Used Avail Use% Mounted on /dev/sda1             8.3G  2.4G  5.5G  30% / /dev/sda2              99M   26M   69M  27% /boot /dev/sdc1              16G   13G  2.3G  85% /export none                  250M     0  250M   0% /dev/shm /dev/sdb1              46G   33M   44G   1% /media [root@roswell media]#

Saturday, October 23, 2010