Linux

Using Google Authenticator for Two Step Auth with SSH

Posted on by looke

If you run servers that are accessible from the internet, you might have noticed the many many brute-force login attempts to random accounts against your system. While the risk, that one of these brute-force attempts succeeds is very low (if you chose decent passwords), one might come to the conclusion, that simple user- and password authentication is not safe enough for complete peace of mind.

Here is where two-factor auth comes to play. Until now, there were not so much options if you didn’t want to spend any money. Here is where Google Authenticator comes in handy: It makes two-factor auth accessible for the greater public.
It consists of two components:

  • An app for your smartphone, that spits out verification codes
  • A PAM module for you linux box, to validate the verification codes

In this article, I will explain how to get and install the Google auth PAM module for Linux.

First, you have to prepare a build environment by fetching all needed development packages:


root@srv /home/me # apt-get install make libpam0g-dev
...
The following NEW packages will be installed:
  binutils cpp cpp-4.4 gcc gcc-4.4 libc-dev-bin libc6-dev libgmp3c2 libgomp1 libmpfr4 libpam0g-dev
  linux-libc-dev make manpages-dev
...

Then, go and grab the source of the Google Authenticator PAM module from http://code.google.com/p/google-authenticator/downloads/list.

After you got the sources and extracted them from the tarball, you can continue with building the module:


root@srv /home/me # cd libpam-google-authenticator-1.0
root@srv /home/me/libpam-google-authenticator-1.0 # make
...
root@srv /home/me/libpam-google-authenticator-1.0 # make install

Install the module and setup PAM to use it for SSH logins:


cp pam_google_authenticator.so /lib/security
cp google-authenticator /usr/local/bin
vim /etc/pam.d/sshd
...
auth       required     pam_google_authenticator.so
...

Setup SSHD to ask for the verification codes:


vim /etc/ssh/sshd_config
...
ChallengeResponseAuthentication yes
...

Setup the module:


root@srv /home/me/libpam-google-authenticator-1.0 # su me

me@srv:~/libpam-google-authenticator-1.0$ /usr/local/bin/google-authenticator

Do you want authentication tokens to be time-based (y/n) y
 ...
Your new secret key is: ...
Your verification code is ...
Your emergency scratch codes are:
 ...

Do you want me to update your "/home/me/.google_authenticator" file (y/n) y

Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y

By default, tokens are good for 30 seconds and in order to compensate for
possible time-skew between the client and the server, we allow an extra
token before and after the current time. If you experience problems with poor
time synchronization, you can increase the window from its default
size of 1:30min to about 4min. Do you want to do so (y/n) y

If the computer that you are logging into isn't hardened against brute-force
login attempts, you can enable rate-limiting for the authentication module.
By default, this limits attackers to no more than 3 login attempts every 30s.
Do you want to enable rate-limiting (y/n) y

After you did everything correctly, you can try to login via SSH and the system should ask you for your verification code:


login as: me
Using keyboard-interactive authentication.
Verification code:
Using keyboard-interactive authentication.
Password:

To round up your setup, you should not forget to remove the packages you installed to build the Google Authenticator module:


root@srv /home/me/libpam-google-authenticator-1.0 # apt-get autoremove make libpam0g-dev

Links
http://code.google.com/p/google-authenticator/

Sending Zabbix Alert SMS via USB modem

Posted on by looke

During some Zabbix sessions, I thought it would be nice to be able to alert via SMS. Zabbix, out of the box, supports the possibility to send SMS via attached GSM modems, so I gave it a try. I am currently using a Huawei USB modem:


Bus 003 Device 011: ID 12d1:1003 Huawei Technologies Co., Ltd. E220 HSDPA Modem / E230/E270/E870 HSDPA/HSUPA Modem

Unfortunately, this modem has some troubles with the AT command sequences Zabbix sends:
/var/log/zabbix-server/zabbix_server.log


   856:20120120:170920.965 Read from GSM modem [^MOK^M]
   856:20120120:170920.965 End of read_gsm():SUCCEED
   856:20120120:170920.965 Write to GSM modem [ATE0^M]
   856:20120120:170920.965 In read_gsm() [OK] [NULL] [NULL] [NULL]
   856:20120120:170921.069 Read from GSM modem [^MOK^M]
   856:20120120:170921.069 In check_modem_result()
   856:20120120:170921.069 End of check_modem_result():SUCCEED
   856:20120120:170921.069 End of read_gsm():SUCCEED
   856:20120120:170921.069 Write to GSM modem [AT^M]
   856:20120120:170921.069 In read_gsm() [OK] [NULL] [NULL] [NULL]
   856:20120120:170921.173 Read from GSM modem [^MOK^M]
   856:20120120:170921.174 In check_modem_result()
   856:20120120:170921.174 End of check_modem_result():SUCCEED
   856:20120120:170921.174 End of read_gsm():SUCCEED
   856:20120120:170921.174 Write to GSM modem [AT+CMGF=1^M]
   856:20120120:170921.174 In read_gsm() [OK] [NULL] [NULL] [NULL]
   856:20120120:170921.277 Read from GSM modem [^MOK^M]
   856:20120120:170921.277 In check_modem_result()
   856:20120120:170921.277 End of check_modem_result():SUCCEED
   856:20120120:170921.277 End of read_gsm():SUCCEED
   856:20120120:170921.277 Write to GSM modem [AT+CMGS="]
   856:20120120:170921.277 Write to GSM modem [0041791234567]
   856:20120120:170921.277 Write to GSM modem ["^M]
   856:20120120:170921.277 In read_gsm() [> ] [NULL] [NULL] [NULL]
   856:20120120:170921.385 Read from GSM modem [^M> ]
   856:20120120:170921.385 In check_modem_result()
   856:20120120:170921.385 End of check_modem_result():SUCCEED
   856:20120120:170921.385 End of read_gsm():SUCCEED
   856:20120120:170921.385 Write to GSM modem [Host xyz is unreachable: PROBLEM]
   856:20120120:170921.385 Write to GSM modem [^Z]
   856:20120120:170921.385 In read_gsm() [+CMGS: ] [NULL] [NULL] [NULL]
   856:20120120:170921.489 Read from GSM modem [^M]
   856:20120120:170921.489 In check_modem_result()
   856:20120120:170921.489 End of check_modem_result():FAIL
   856:20120120:170921.489 End of read_gsm():FAIL
   856:20120120:170921.489 Write to GSM modem [^MESC^Z]
   856:20120120:170921.489 In read_gsm() [] [NULL] [NULL] [NULL]
   856:20120120:170921.489 Error during wait for GSM modem.
   856:20120120:170921.489 Read from GSM modem []
   856:20120120:170921.489 End of read_gsm():SUCCEED
   856:20120120:170921.494 End of send_sms():FAIL
   856:20120120:170921.494 End execute_action()
   856:20120120:170921.494 Error sending alert ID [62]

After some research I figured out that it probably would be a better idea to write a wrapper-script to implement the SMS functionality. There is actually a way to fix this AT command sequence issue, but it would require recompiling some parts of Zabbix (which is not an option for me, as I use the Debian packaged Zabbix). To interface with the modem, I am finally using Gnokii:

/etc/zabbix/gnokii.conf


[global]
port = /dev/ttyUSB1
model = AT
connection = serial

Thats the script I use to send the alerts to (taken straight from zabbix.com):

/etc/zabbix/alert.d/zabbix-sms.sh


#!/bin/sh 
LOGFILE="/var/log/zabbix-server/zabbix-sms.log" 
echo "To: '$1' Text: '$3'" >> ${LOGFILE} 
PHONENR=`echo "$1" | sed s#\s##` 
/bin/echo "$3" | /usr/bin/gnokii --config /etc/zabbix/gnokii.conf --sendsms "${PHONENR}" 1>>${LOGFILE} 2>&1

Here are some screenshots on how to configure the SMS alert in the Zabbix GUI:

Links
http://www.zabbix.com/wiki/howto/config/alerts/sms
http://lab4.org/wiki/Zabbix_Medien_einrichten

Mounting LVM nested Partitons

Posted on by looke

With this post, I want to describe how you can mount partitions, nested within LVM volumes. A possible use-case includes file based backup of virtual machines running on LVM volumes.

In the example below, I use a Windows Server 2008 system partition (NTFS). The Windows server is running on a Debian KVM system.

You will need:

  • kpartx
  • (In this exampe also ntfs-3g)

Lets get started – first some information about the LVM setup:


root@SERVER:/# lvdisplay
  --- Logical volume ---
  LV Name                /dev/lvm/windoze2k8
  VG Name                lvm
  LV UUID                836UYu-lmuT-qCUg-2lRx-QNgZ-COf7-h6NUH6
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                19.53 GiB
  Current LE             5000
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:1

The partition table of our LVM volume:


root@SERVER:/# fdisk -l /dev/lvm/windoze2k8

Disk /dev/lvm/windoze2k8: 21.0 GB, 20971520000 bytes
255 heads, 63 sectors/track, 2549 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x1a13ce21

                       Device Boot      Start         End      Blocks   Id  System
/dev/lvm/windoze2k81   *           1        2550    20477952    7  HPFS/NTFS

Now, lets move on to the the actual doing:


root@SERVER:~# kpartx -a /dev/lvm/windoze2k8
root@SERVER:~# mkdir /mnt/lvm-windoze2k81 && mount /dev/mapper/lvm-windoze2k81 /mnt/lvm-windoze2k81

Et voila:


root@SERVER:~# ls /mnt/lvm-windoze2k81
autoexec.bat  bootmgr       config.sys              hiberfil.sys  PerfLogs     Program Files  System Volume Information  Windows
Boot          BOOTSECT.BAK  Documents and Settings  pagefile.sys  ProgramData  $RECYCLE.BIN   Users

To remove the mapping, do the following:


root@SERVER:/# umount /mnt/lvm-windoze2k81
root@SERVER:/# kpartx -d /dev/lvm/windoze2k8

Note
If you want to use such a setup to do file based backup of running virtual machines, it is wise to create a LVM snapshot first and applying kpartx on the snapshot device.

List based permanent bans with fail2ban

Posted on by looke

Today I post something about the nice little tool fail2ban. As you probably know, fail2ban can be used to block those annoying brute force attacks against your servers. Other than the also popular and useful tool DenyHosts it allows the protection of other services than SSH as well (e.g. HTML login pages served by Apache). The working mechanism also differs from that of DenyHosts, as fail2ban uses iptables instead of the BSD style hosts.deny file to block annoying brute forcers. Installation is quite simple, on Debian for example, just install it through apt and you’re good to go even with the default config.

One thing that I was missing, was the option to ban IPs forever. You can basically do this by setting bantime to a negative value, but as soon as you reload your iptables rules (e.g. by restarting the fail2ban service or the whole system) the entries for the permanently banned IPs are gone.
To overcome this issue, I did some minor changes to the actions fail2ban executes on start-up and on banning.

IMPORTANT: I strongly advise you, to be careful while playing around with automated banning tools, especially if you can’t reach your server physically. Make sure, that you have something useful set in the ignoreip option under the [DEFAULT] jail (your current IP address) to not accidentally lock you out of the system (really nasty with permanent banning active…)

  1. First, check the banaction currently used (you need that, to modify the correct actionfile afterwards)
    /etc/fail2ban/jail.local

    
#
# ACTIONS
#
...
banaction = iptables-multiport
...
  2. Open up the corresponding actionfile and modify according to the sample below (changes are under the # Persistent banning of IPs comment)
    /etc/fail2ban/action.d/iptables-multiport.conf

    
...
actionstart = iptables -N fail2ban-<name>
              iptables -A fail2ban-<name> -j RETURN
              iptables -I INPUT -p <protocol> -m multiport --dports <port> -j fail2ban-<name>
              # Persistent banning of IPs
              cat /etc/fail2ban/ip.blacklist | while read IP; do iptables -I fail2ban-<name> 1 -s $IP -j DROP; done
...
actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
            # Persistent banning of IPs
            echo '<ip>' >> /etc/fail2ban/ip.blacklist
...
  3. Your blacklist should look something like this (one IP per line, of course you can add IPs manually)
    /etc/fail2ban/ip.blacklist

    
...
10.0.0.242
192.168.1.39
...
  4. Restart fail2ban to make the changes active

Now, what happens is that each time fail2ban starts, it loops through your ip.blacklist and blocks the IPs in there. If fail2ban blocks a new IP, it will automatically append it to the blacklist.

Links
http://www.fail2ban.org
http://www.fail2ban.org/wiki/index.php/Whitelist
http://denyhosts.sourceforge.net

Update
The following config adds some nice features that were missing in the example above:

  • No duplicate iptables rules (@Lin: might be interesting for you)
  • Jail specific blocking rules (similar to Dr. Tyrell’s and samuelE’s suggestions in the comments)
  • Reporting offender IPs to badips.com

/etc/fail2ban/action.d/iptables-multiport.conf:


# Fail2Ban configuration file
#
# Author: Cyril Jaquier
# Modified by Yaroslav Halchenko for multiport banning and Lukas Camenzind for persistent banning 
#
#
[Definition]
# Option:  actionstart
# Notes.:  command executed once at the start of Fail2Ban.
# Values:  CMD
#
actionstart = iptables -N fail2ban-<name>
              iptables -A fail2ban-<name> -j RETURN
              iptables -I INPUT -p <protocol> -m multiport --dports <port> -j fail2ban-<name>
              # Load local list of offenders
              if [ -f /etc/fail2ban/ip.blacklist ]; then cat /etc/fail2ban/ip.blacklist | grep -e <name>$ | cut -d "," -s -f 1 | while read IP; do iptables -I fail2ban-<name> 1 -s $IP -j DROP; done; fi
# Option:  actionstop
# Notes.:  command executed once at the end of Fail2Ban
# Values:  CMD
#
actionstop = iptables -D INPUT -p <protocol> -m multiport --dports <port> -j fail2ban-<name>
             iptables -F fail2ban-<name>
             iptables -X fail2ban-<name>
# Option:  actioncheck
# Notes.:  command executed once before each actionban command
# Values:  CMD
#
actioncheck = iptables -n -L INPUT | grep -q fail2ban-<name>
# Option:  actionban
# Notes.:  command executed when banning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    <ip>  IP address
#          <failures>  number of failures
#          <time>  unix timestamp of the ban time
# Values:  CMD
#
actionban = if ! iptables -C fail2ban-<name> -s <ip> -j DROP; then iptables -I fail2ban-<name> 1 -s <ip> -j DROP; fi
            # Add offenders to local blacklist, if not already there
            if ! grep -Fxq '<ip>,<name>' /etc/fail2ban/ip.blacklist; then echo '<ip>,<name>' >> /etc/fail2ban/ip.blacklist; fi
            # Report offenders to badips.com
            wget -q -O /dev/null www.badips.com/add/<name>/<ip>
# Option:  actionunban
# Notes.:  command executed when unbanning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    <ip>  IP address
#          <failures>  number of failures
#          <time>  unix timestamp of the ban time
# Values:  CMD
#
actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
              # Disabled clearing out entry from ip.blacklist (somehow happens after each stop of fail2ban)
              # sed --in-place '/<ip>,<name>/d' /etc/fail2ban/ip.blacklist
[Init]
# Defaut name of the chain
#
name = default
# Option:  port
# Notes.:  specifies port to monitor
# Values:  [ NUM | STRING ]  Default:
#
port = ssh
# Option:  protocol
# Notes.:  internally used by config reader for interpolations.
# Values:  [ tcp | udp | icmp | all ] Default: tcp
#

Sorting out denied SMB access

Posted on by looke

Assumption
The fileserver is joined to a ActiveDirectory domain through Winbind

Issue
SMB/Filesystem permissions seem to not apply, if a folder is owned by a local group and the domain users are members of that group.
Observable effects are “Access denied” messages while trying to access the SMB share from a windows machine with a domain user, even though through SSH the domain user can access the respective folder.
A common scenario is, if the file server was recently integrated into a domain and there are still local, non-domain users working on it.

Some information to start with:


[root@fileserver ~]# id user
uid=900(user) gid=1000(localgroup) groups=1000(localgroup)

[root@fileserver ~]# id DOMAIN+user
uid=20000(DOMAIN+user) gid=20000(DOMAIN+domain users) groups=20000(DOMAIN+domain users),1000(localgroup),20001(DOMAIN+domaingroup),10008(BUILTIN+users)

[root@fileserver ~]# ls -la /data
drwxrwxrwx 10 root    root                 4096 Feb 30 13:37 .
drwxr-xr-x 28 root    root                 4096 Feb 30 13:37 ..
...
drwxrwx---  6 root    localgroup           4096 Feb 30 13:37 share
...

[root@fileserver ~]# getent group localgroup
localgroup:x:1000:DOMAIN+user

Solution
Mapping local users to domain users. Check option “username map”

/etc/samba/smb.conf:


[global]
	workgroup = DOMAIN
	realm = DOMAIN.COM
	password server = DC.DOMAIN.COM
	winbind separator = +	
	security = ads
	...	
	username map = /etc/samba/smbusers
	...	
	
[share]
	comment = My share
	browseable = yes
	writeable = yes
	readonly = no
	path = /data/share
	guest ok = no
	create mask = 0770
	directory mask = 0770
	inherit acls = yes
	inherit permissions = yes

/etc/samba/smbusers:


# Unix_name = SMB_name1 SMB_name2 ...
root = administrator admin
nobody = guest pcguest smbguest
user = DOMAIN+user

smb.conf manpage
http://www.samba.org/samba/docs/man/manpages-3/smb.conf.5.html

Application backup – Scripted pre- and postbackup actions

Posted on by looke

Many of today’s businesses rely heavily on their application servers. The times of simple fileshares and single-document based processes are over and with them the time of simple filecopy as a method of backing up is over.

In this article I want to describe a method to backup the two most common components of a modern application service: filesystem and database.
No matter how you solve your backup, the approach should always make sure, that the database integrity is given and that the filesystem is in sync with the database state.

The following two scripts are deployed as a Pre- and a Post backup script. The Pre- Backup script stops the application, dumps the database contents, creates a LVM snapshot and re-starts the application. Post- Backup removes the snapshot.

The fileset for the backup application would then look like this:
Database dumps: /dbdump
Filesystem snapshot: /volume-snapshot

appbackup-run-before.sh


#!/bin/bash
#
# Application Service Backup - Part 1 of 2
#
# Pre-Backup script
# - Stops Service
# - Takes a MySQL Dump
# - Creates a LVM Snapshot
# - Restarts Service
# - Mounts the LVM Snapshot
#
# 2010, Looke
#

# Which service to mess with
SERVICE="service"

# LVM Stuff
LVMVOLUME="/dev/lvm/volume"
LVMSNAPSHOT="volume-snapshot"
LVMSNAPSHOTSIZE="50G"

# MySQL Properties
DBDUMPDIR="/dbdump"
DBNAME="aaa"
DBHOST="zzz"
DBUSER="xxx"
DBPASSWORD="yyy"

echo "Shutting down Service..."
/etc/init.d/${SERVICE} stop

while ps ax | grep -v grep | grep ${SERVICE} > /dev/null;
do
  echo "...stopping..."
  sleep 5
done

echo "Creating MySQL Dump..."
if [ ! -d "${DBDUMPDIR}" ]; then
  mkdir -p ${DBDUMPDIR}
fi
mysqldump --host=${DBHOST} --user=${DBUSER} --password=${DBPASSWORD} ${DBNAME} > ${DBDUMPDIR}/${DBNAME}.sql

echo "Creating LVM Snapshot..."
modprobe dm-snapshot
lvm lvcreate --size ${LVMSNAPSHOTSIZE} --snapshot --name ${LVMSNAPSHOT} ${LVMVOLUME}
sleep 5

echo "Restarting Service..."
/etc/init.d/${SERVICE} start

while ! ps ax | grep -v grep | grep ${SERVICE} > /dev/null;
do
  echo "...starting..."
  sleep 5
done

echo "Mounting LVM Snapshot..."
if [ ! -d "/${LVMSNAPSHOT}" ]; then
  mkdir -p /${LVMSNAPSHOT}
fi
mount -o ro /dev/lvm/${LVMSNAPSHOT} /${LVMSNAPSHOT}

exit 0

appbackup-run-after.sh


#!/bin/bash
#
# Application Service Backup - Part 2 of 2
#
# Post-Backup script
# - Unmounts the LVM Snapshot
# - Destroys the LVM Snapshot
#
# 2010, Looke
#

# LVM Stuff
LVMSNAPSHOT="volume-snapshot"

echo "Unmounting LVM Snapshot..."
umount /${LVMSNAPSHOT}

echo "Destroying LVM Snapshot..."
lvm lvremove -f /dev/lvm/${LVMSNAPSHOT}

exit 0

One backdraw of this method is, that the service has to be stopped in order to get a consistent state of the data. If the service has to be online 24/7 you would have to consider clustering (anyways, you would have to come up with something to cover unplanned downtimes).

Here is a small excerpt to show you how to configure the Pre- and Post backup scripts with the open source backup software Bacula. I assume if you use some other backup software, you can click your way through the GUI yourself :)



Job {
  Name = "Appbackup"
  ...
  Client Run Before Job = "/opt/bacula/scripts/appbackup-run-before.sh"
  Client Run After Job = "/opt/bacula/scripts/appbackup-run-after.sh"
  ...
}

Useful links:
Bacula Documentation – Job Ressource
Ubuntuusers Wiki – LVM (german)

Moving a XEN guest to a new Dom0

Posted on by looke

I assume, you use LVM volumes for your XEN guests. I’m not going to use “xm migrate” here, the method used works by dd’ing the LVM volume over to the new Dom0, so make sure, you have a fitting LVM volume in place on your destination system.
I recommend you to stop the machine you’re going to move (or you could consider to create a LVM snapshot). Anyways, if you know nothing will change you can try it with the running machine (I did this once, and it resulted in a fsck upon boot but without any further problems).

With this one you can dd the LVM volume to the new host:


dd if=/dev/x bs=1M | ssh username@remote-server "dd of=/dev/y bs=1M"

To check the status of the copyjob, open a new console and issue (note: the USR1 signal lets dd print some infos):


watch -n 5 "killall -USR1 dd"

To finish the move, copy the XEN host config file to the new system:


scp /etc/xen/hostconfig username@remote-server:/etc/xen/hostconfig

Links
http://en.wikipedia.org/wiki/Dd_(Unix)

Monitoring a remote network interface with tcpdump and Wireshark

Posted on by looke

In this small how-to, I’ll show how to capture network traffic from a remote system to analyze it using Wireshark.

All you need is tcpdump on the remote machine, where you want to dump the network traffic off and Wireshark on the computer, you want to use to look at the packets flying around.
I use this setup for checking, whats going on on my IPcop firewall.

First, you need to prepare a named pipe on you monitoring station:


mkfifo /tmp/pipe

After this, we build up the connection to the remote system, issue the tcpdump command there and direct all outputs to the pipe:


ssh root@10.1.1.254 "tcpdump -i eth0 -s 0 -U -w - not port 22" > /tmp/pipe

Now switch to another console and start Wireshark, listening to our newly created pipe:


wireshark -k -i /tmp/pipe

After Wireshark has started, the ssh console will ask for roots password. After you entered it, you will see the packets getting listed in Wiresharks main screen.

Used tcpdump options

  • -i eth0 specifies the interface to capture from (change to your needs)
  • -s 0 sets the packet snapshot lenght it to the default of 65535, for backwards compatibility with recent older versions of tcpdump
  • -U writes each incoming packet to the file (or std. out) immediately, instead of waiting until the buffer has filled
  • -w – writes to standard output
  • not port 22 keeps tcpdump from returning the traffic we create with our ssh connection

Further info
http://wiki.wireshark.org/CaptureSetup/Pipes
http://www.tcpdump.org/tcpdump_man.html

Deploying the open-source backup solution Bacula

Posted on by looke

It’s now about two years ago, that I wondered “Why the … are we paying support and license subscriptions, if the only benefit is that you can listen to the support line music and get a new logo in the softwares main window after each update?”. Ok, the software works so far. But for every new client, you have to relicense and especially support for linux hosts can be a real pain.

I don’t want to call any names here nor start an argument with any fanboys. But being tired of all this commercial “corporate” softwares, I want to share my approach to installing the free and open source backup software Bacula.

Please feel free, to write me if you find possible errors or misconfigurations. I plan to extend this how-to with more detailled instructions.

Well, back to bacula: This overview visualizes the interactions of all bacula modules (taken from the bacula.org wiki)

To keep things simple, I start with a small but expandable test installation, consisting of one server and one or maybe two clients. In this case:

Hosts

  • bacula-server (Debian Lenny) – Director, Storage Daemon, File Daemon
  • mysql-server – MySQL Catalog
  • bacula-client-linux (Debian Lenny) – File Daemon
  • bacula-client-win (WinXP) – File Daemon

Following, I note all commands that are necessary to install the mentioned scenarion.

Installation of bacula-server (Director, Storage Daemon, File Daemon)


bacula-server:~# aptitude install build-essential libpq-dev libncurses5-dev libssl-dev psmisc libmysqlclient-dev mysql-client
bacula-server:~# cd /usr/local/src
bacula-server:~# wget http://downloads.sourceforge.net/project/bacula/bacula/5.0.1/bacula-5.0.1.tar.gz
bacula-server:~# tar xzvf bacula-5.0.1.tar.gz
bacula-server:~# cd bacula-5.0.1

To simplify the configure process, I used a shellscript with all the options (also the ones recommended by the Bacula project)


#!/bin/sh
prefix=/opt/bacula
CFLAGS="-g -O2 -Wall" \
  ./configure \
    --sbindir=${prefix}/bin \
    --sysconfdir=${prefix}/etc \
    --docdir=${prefix}/html \
    --htmldir=${prefix}/html \
    --with-working-dir=${prefix}/working \
    --with-pid-dir=${prefix}/working \
    --with-subsys-dir=${prefix}/working \
    --with-scriptdir=${prefix}/scripts \
    --with-plugindir=${prefix}/plugins \
    --libdir=${prefix}/lib \
    --enable-smartalloc \
    --with-mysql \
    --enable-conio \
    --with-openssl \
    --with-smtp-host=localhost \
    --with-baseport=9101 \
    --with-dir-user=bacula \
    --with-dir-group=bacula \
    --with-sd-user=bacula \
    --with-sd-group=bacula \
    --with-fd-user=root \
    --with-fd-group=bacula

Paste the code above in a file, make it executable (chmod +x) and run it.

If everything worked fine, type:


bacula-server:~# make && make install

Now to the setup of baculas catalog database. In my case, I use MySQL as catalog background, because I already have some knowledge about it. Other databases are supported as well (i.e. Postgres).
Bacula comes with all necessary scripts to create the initial catalog database on a local MySQL instance (I recommend you to apt-get the MySQL server and leave the root PW empty during the bacula setup phase). To have it setup on a remote server, you just need to check out the scripts, strip away the shell stuff and copy&paste the statements to your DB server (Thats what I did).


bacula-server:~# groupadd bacula
bacula-server:~# useradd -g bacula -d /opt/bacula/working -s /bin/bash bacula
bacula-server:~# passwd bacula
bacula-server:~# chown root:bacula /opt/bacula
bacula-server:~# chown bacula:bacula /opt/bacula/working
bacula-server:~# mkdir /backup2disk && chown -R bacula:bacula /backup2disk
bacula-server:~# touch /var/log/bacula.log && chown bacula:bacula /var/log/bacula.log
bacula-server:~# chown bacula:bacula /opt/bacula/scripts/make_catalog_backup /opt/bacula/scripts/delete_catalog_backup
bacula-server:~# cp /opt/bacula/scripts/bacula-ctl-dir /etc/init.d/bacula-dir
bacula-server:~# cp /opt/bacula/scripts/bacula-ctl-sd /etc/init.d/bacula-sd
bacula-server:~# cp /opt/bacula/scripts/bacula-ctl-fd /etc/init.d/bacula-fd
bacula-server:~# chmod 755 /etc/init.d/bacula-*
bacula-server:~# update-rc.d bacula-sd defaults 91
bacula-server:~# update-rc.d bacula-fd defaults 92
bacula-server:~# update-rc.d bacula-dir defaults 90

The following configfiles contain my example config (rename bacula-server-bacula-fd.conf to bacula-fd.conf):

bacula-dir.conf
bacula-sd.conf
bacula-server-bacula-fd.conf
bconsole.conf

Installation of Bweb and Brestore on bacula-server

If you like to actually see whats happening with your backups whithout hacking away on the console, I recommend you to install Bweb.


bacula-server:~# aptitude install lighttpd ttf-dejavu-core libgd-graph-perl libhtml-template-perl libexpect-perl libdbd-pg-perl libdbi-perl libdate-calc-perl libtime-modules-perl
bacula-server:~# /etc/init.d/lighttpd stop
bacula-server:~# update-rc.d -f lighttpd remove
bacula-server:~# cd /var/www
bacula-server:~# wget http://downloads.sourceforge.net/project/bacula/bacula/5.0.1/bacula-gui-5.0.1.tar.gz
bacula-server:~# tar xzvf bacula-gui-5.0.1.tar.gz
bacula-server:~# ln -s /var/www/bacula-gui-5.0.1 /var/www/bacula-gui
bacula-server:~# cd /var/www/bacula-gui/bweb

This is my httpd.conf, which contains logging and authentication support:
bweb-httpd.conf


bacula-server:~# touch /var/log/lighttpd/access.log /var/log/lighttpd/error.log
bacula-server:~# chown -R bacula:bacula /var/log/lighttpd
bacula-server:~# ln -s /opt/bacula/bin/bconsole /usr/bin/bconsole
bacula-server:~# chown bacula:bacula /opt/bacula/bin/bconsole /opt/bacula/etc/bconsole.conf
bacula-server:~# chown -R bacula:bacula /var/www/bacula*
bacula-server:~# cd /var/www/bacula-gui/bweb/script
bacula-server:~# mysql -p -u bacula -h mysql-server bacula < bweb-mysql.sql
bacula-server:~# ./starthttp

After we start lighttpd for the first time, it creates the bweb.conf configfile, which we own to the bacula user:


bacula-server:~# chown bacula:bacula /var/www/bacula-gui/bweb/bweb.conf

Now, open up a browser and navigate to the bweb page (lighttpd tells you where you can reach it after you start the service). Check out the following screenshot to see how to configure the Bweb instance:

If you also like to run restore jobs in a graphical manner, you can install the Brestore addon to your new Bweb interface.


bacula-server:~# aptitude install libdbd-pg-perl libexpect-perl libwww-perl libgtk2-gladexml-perl unzip
bacula-server:~# cd /var/www/bacula-gui/brestore
bacula-server:~# mkdir -p /usr/share/brestore
bacula-server:~# install -m 644 -o root -g root brestore.glade /usr/share/brestore
bacula-server:~# install -m 755 -o root -g root brestore.pl /usr/bin
bacula-server:~# cd /var/www/bacula-gui/bweb/html
bacula-server:~# wget http://www.extjs.com/deploy/ext-3.1.1.zip
bacula-server:~# unzip ext-3.1.1.zip
bacula-server:~# rm ext-3.1.1.zip
bacula-server:~# mv ext-3.1.1 ext
bacula-server:~# chown -R bacula:bacula ext
bacula-server:~# nano /etc/mime.types

Add a new MIME type:


text/brestore                                   brestore.pl

Restart the lighttpd server:


bacula-server:~# killall lighttpd
bacula-server:~# /var/www/bacula-gui/bweb/script/starthttp

Installation of bacula-client-linux (File Daemon)

I assume, you have a Debian Lenny system up and running.


bacula-client-linux:~# aptitude install build-essential libssl-dev
bacula-client-linux:~# cd /usr/local/src
bacula-client-linux:~# wget http://downloads.sourceforge.net/project/bacula/bacula/5.0.1/bacula-5.0.1.tar.gz
bacula-client-linux:~# tar xzvf bacula-5.0.1.tar.gz
bacula-client-linux:~# cd bacula-5.0.1

I also use a shellscript to configure our File Daemon, to make it more comfortable to deploy on multiple clients.


#!/bin/sh
prefix=/opt/bacula
CFLAGS="-g -O2 -Wall" \
  ./configure \
    --sbindir=${prefix}/bin \
    --sysconfdir=${prefix}/etc \
    --docdir=${prefix}/html \
    --htmldir=${prefix}/html \
    --with-working-dir=${prefix}/working \
    --with-pid-dir=${prefix}/working \
    --with-subsys-dir=${prefix}/working \
    --with-scriptdir=${prefix}/scripts \
    --with-plugindir=${prefix}/plugins \
    --libdir=${prefix}/lib \
    --enable-smartalloc \
    --with-openssl \
    --enable-client-only


bacula-client-linux:~# make && make install
bacula-client-linux:~# cp /opt/bacula/scripts/bacula-ctl-fd /etc/init.d/bacula-fd
bacula-client-linux:~# chmod 755 /etc/init.d/bacula-fd
bacula-client-linux:~# update-rc.d bacula-fd defaults 90

Finally, the configfile for our linux client (rename bacula-client-linux-bacula-fd.conf to bacula-fd.conf):

bacula-client-linux-bacula-fd.conf

Installation of bacula-client-win (File Daemon)

Get the windows binaries from the bacula page and make your way through the install dialogue:

Testing

Starting the Bacula services on bacula-server:


bacula-server:~# /etc/init.d/bacula-sd start
bacula-server:~# /etc/init.d/bacula-fd start
bacula-server:~# /etc/init.d/bacula-dir start

Starting the File Daemon on bacula-client-linux


bacula-client-linux:~# /etc/init.d/bacula-fd start

Bconsole Commands on bacula-server


bconsole
status
list clients
quit

Extended scenario – Tapelibrary on a separate server called “bacula-storage”
In this case, you don’t need to build the whole package. Apt-get the same packages as mentioned in the installation of bacula-server, get the bacula tarball, unpack and configure with the following script:

#!/bin/sh
prefix=/opt/bacula
CFLAGS="-g -O2 -Wall" \
  ./configure \
    --sbindir=${prefix}/bin \
    --sysconfdir=${prefix}/etc \
    --docdir=${prefix}/html \
    --htmldir=${prefix}/html \
    --with-working-dir=${prefix}/working \
    --with-pid-dir=${prefix}/working \
    --with-subsys-dir=${prefix}/working \
    --with-scriptdir=${prefix}/scripts \
    --with-plugindir=${prefix}/plugins \
    --libdir=${prefix}/lib \
    --enable-smartalloc \
    --with-mysql \
    --with-openssl \
    --with-smtp-host=localhost \
    --with-baseport=9101 \
    --disable-build-dird \
    --with-sd-user=bacula \
    --with-sd-group=bacula \
    --with-fd-user=root \
    --with-fd-group=bacula

2be continued with
– Bweb ssh remote command execution to show library status (reminder: don’t forget chmod g-w /opt/bacula/working)
– Extended configfiles

Hints
An Issue, that I noticed was, that brestore didn’t allow you to graphically drill down to the files you wanted to restore. You couldn’t click your way through the path but had to enter the path to the desired file by hand. It seems, that as soon as you back up another host, this problem resolves itself.

Links

Main manual: http://www.bacula.org/5.0.x-manuals/en/main/main/index.html

Ubuntu 9.10 and Windows 7 dualboot with a Fakeraid Controller

Posted on by looke

I recently tried to install Ubuntu Server 9.10 and Windows 7 in a dual boot configuration on a Promise FastTrak TX2300 SATA RAID1 array and unexpectedly ran into some problems.
It seems that the FastTrak TX2300 SATA RAID controller doesn’t have fully featured RAID options (see Fakeraid: https://help.ubuntu.com/community/FakeRaidHowto) an therefore needs a bit a different approach to make it do what you want.

Here I provide a manual on how I got my stuff working.

Installation of Ubuntu

  • Boot from the Ubuntu 9.10 Server install CD and start the setup
  • In order to use fakeraid arrays, we need the dmraid package; Ubuntu 9.10 already has it included.
  • Partition as usual (e.g. root filesystem with ext4, empty partition (for later Windows 7 installation), swap at end of harddrive)
  • GRUB installation will skip because it can’t write to the MBR, therefore pick “Continue without bootloader” from the installer menue

At this point, the system is not ready to boot. We will handle this later.
First, we continue with Windows 7.

Installation of Windows 7

  • Make your way through the installation and select your Windows 7 partition as installation target.
  • In case the created partition for Windows 7 should not be accepted as valid installation target, you can press Shift-F10 and use the “diskpart” utility to delete and recreate the Windows 7 partition (diskpart, list disk, select disk x, list partition, delete partition x, create partition)

After installing Windows 7, the system is usable (at least 50% of it), allowing you to boot into Windows 7. To be able to select between Ubuntu and Windows, we need to install GRUB, which we skipped in Step 1 and configure it accordingly.

Setting up GRUB

  • Boot from the Ubuntu 9.10 Server CD and enter the “Rescue a broken system” mode. As soon as you get to the Rescue mode switch to another console (e.g. Alt-F2)

mount /dev/mapper/pdc_bfihaijgha1 /mnt (replace with the name of your mapped Ubuntu root partition)
mount --bind /dev /mnt/dev/
mount -t proc proc /mnt/proc/
mount -t sysfs sys /mnt/sys/
chroot /mnt /bin/bash

In my case, I had to enable the CD-ROM as apt package source because I didn’t have a network connection on that computer:


nano /etc/apt/sources.list

and uncomment the line beginning with


"#deb cdrom:[Ubuntu-Server 9.10...."

Now you can install GRUB from the CD-ROM and set it up


apt-get install grub
cp /usr/lib/grub/i386-pc/* /boot/grub/

grub
grub> device (hd0) /dev/mapper/pdc_bfihaijgha (replace with the name of your mapped RAID volume)
grub> find /boot/grub/stage1
grub> root (hd0,0)
grub> setup (hd0)
grub> quit

update-grub (the menu.lst gets created for you)

Add the Windows 7 entry to menu.lst:


nano /boot/grub/menu.lst

and add the following lines to the bottom of the file


title Windows 7
rootnoverify (hd0,1)
makeactive
chainloader +1

here you can also adjust the bootmenue to show up without first pressing “Esc” (comment hidemenue) and change timers. After you have changed everything to your needs, you can restart the system and check if the bootmenue displays everything correctly and if all entries are working.