Tinkering with Linux and Electronics

Configuring your DHCP server for iPXE booting

NOTE: This is Part 5 in a series on booting Ubuntu from an iSCSI disk. View all of the sections here.

Previously we setup a iSCSI target (server) and installed Ubuntu to the iSCSI disk. We also setup iPXE on the server, but we need to configure dnsmasq to route the client correctly.

In my previous post I walked through how the iPXE process flows. Here’s a quick overview to refresh your memory:

We already have the server configured so now let’s work through the roles of the DHCP server.

1. The DHCP server needs to respond to DHCP requests with the TFTP server information. Any clients not concerned with iSCSI booting will ignore the extra information while our iSCSI initiator will use it to download and run the iPXE executable.
2. The iPXE executable, when run, makes an additional, special DHCP request that includes a flag indicating that the request was made from an iPXE executable. When the DHCP server receives this type of request, it needs to respond with the iSCSI connection information, rather than the TFTP info.

Much of what follows is already documented at http://ipxe.org/howto/chainloading, but to be throrough, the two most popular DHCP servers are dhcpd and dnsmasq

ISC DHCPD

Note: I don’t have a server running dhcpd at this time so this section is entirely hearsay until I learn otherwise.

We need to edit /etc/dhcpd.conf to contain

next-server 192.168.0.101;
filename "undionly.kpxe";

Where 192.168.0.101 is the IP of the TFTP server you setup in Part 4. That takes care of roll #1 of the DHCP server. You should now be able to test your progress by booting your client machine. You should see iPXE start, request an IP address, which will cause it to re-run iPXE in an infinite loop. That’s a good thing… for now.

To break out of the cycle, also add the following to your /etc/dhcpd.conf file

if exists user-class and option user-class = "iPXE" {
    filename "iscsi:192.168.0.101::::iqn.2012-03.com.example:xbmc-client";
} else {
    filename "undionly.kpxe";
}

Where 192.168.0.101 is your iSCSI target. Again, find lots of useful information here.

That takes care of the DHCP server’s second role. If everything is configured properly, you’re done! Grab a cold one and watch as your system follows the boot process outlined above to load GRUB, which in turn boots Ubuntu!

DNSMASQ

To configure DNSMASQ, we’ll need to edit /etc/dnsmasq.conf

Side note: If you’re using untangle as your DHCP server, you can click Config > Networking > Advanced > DHCP & DNS. Any text entered in the text box here will be magically added to dnsmasq.conf and the dnsmasq service will be restarted for you when you click the Apply button.

First we need to respond to standard DHCP requests with the TFTP information

enable-tftp
tftp-root=/mnt/media/tftpboot

dhcp-boot=tag:!IPXEBOOT,undionly.kpxe,TFTP_Server_Name,192.168.0.101

The first two lines should be pretty self explanitory while the last line requires some explanation. We are specifying a conditional boot option. The first section declares the condition. If the request was NOT tagged with IPXEBOOT, then specify the client the filename (undionly.kpxe), TFTP server name and IP address where that file can be found. If the TFTP server and DHCP server are the same box, you can omit the last two parameters like so

dhcp-boot=tag:!IPXEBOOT,undionly.kpxe

Important Note: If your clients hang at this point, you may, (like me) have an older version of dnsmasq. Older version of dnsmasq used “net:” to instead of “tag:” and they used # instead of ! to negate the result. So my configuration looks like this:

enable-tftp
tftp-root=/mnt/media/tftpboot

dhcp-boot=net:#IPXEBOOT,undionly.kpxe,TFTP_Server_Name,192.168.0.101
#In the previous line, the # symbol is not a comment, it is a negation operator.
#Meaning that the options specified here will only apply to requests NOT
#tagged with IPXEBOOT

dhcp-match=IPXEBOOT,175
dhcp-option=175,8:1:1
dhcp-option=tag:IPXEBOOT,17,"iscsi:192.168.0.101::::iqn.2012-03.com.example:xbmc-server"

• Line 1 checks if dhcp-option 175 (iPXE/gPXE related option) was specified by the client, and if so, it tags the request with IPXEBOOT.
• Line 2 sets option number 175 to 8:1:1 which effectively enables the KEEP_SAN_ALIVE parameter which is required for iSCSI booting.
• Line 3 once again sets a conditional boot option. If the request has been tagged with the “IPXEBOOT” tag, then specify the iSCSI disk as the path name to the client’s root disk (i.e. option 17).

Once more, since my dnsmasq does not support the “tag:” syntax, my dnsmasq.conf contains the following:

dhcp-match=IPXEBOOT,175
dhcp-option=175,8:1:1
dhcp-option=net:IPXEBOOT,17,"iscsi:192.168.0.101::::iqn.2012-03.com.example:xbmc-server"

That takes care of the DHCP server’s second role. If everything is correct, you’re done! Grab a cold one and watch as your system follows the boot process outlined above to load GRUB, which in turn boots Ubuntu!

For more information on the dhcp-options available to dnsmasq, see http://www.ietf.org/rfc/rfc2132.txt

Configuring iPXE (formerly gPXE) for booting an iSCSI disk

NOTE: This is Part 4 in a series on booting Ubuntu from an iSCSI disk. View all of the sections here.

In my previous posts I’ve walked through how to install and configure an Ubuntu installation on an iSCSI disk. We now need to setup our bootloaders to connect to and boot from the iSCSI disk. The bootloader of choice is iPXE (formerly gPXE). If you’re not familiar with iPXE:

iPXE is the leading open source network boot firmware. It provides a full PXE implementation enhanced with additional features - ipxe.org

Essentially, just like PXE, it allows you to get the files needed to boot from another computer on your network. However, iPXE provides additional protocols that normal PXE booting doesn’t. There are a number of different ways to setup your computer to start iPXE.

1. The most straightforward method is to flash your network card with an iPXE ROM. This way your BIOS/EFI can run iPXE directly via the ethernet card. (see http://ipxe.org/howto/romburning for more information)
2. The easiest method is to copy iPXE to local storage such as a CD, USB or even floppy drive.
3. Since I wasn’t interested in flashing my card and didn’t have a spare USB disk to donate to the cause, I chose to PXE boot into iPXE. This is a bit more convoluted than the other options, but it allows me to run completely diskless on my client machine.

PXE/iPXE Overview & Differences

Here’s an overview of how standard PXE booting works.

1. When you PXE boot a computer, the network card makes a DHCP request to the network.
2. The DHCP server gets configured to respond to DHCP requests with not only an address, but also the TFTP server name, and the file name of the PXE executable.
3. When the client receives this information, it connects to the specified TFTP server to download and run the PXE executable.
4. The PXE executable connects to the TFTP server again and looks for the default menu configuration file on the server.
5. Finally, the menu allows you to select a number of boot options a la GRUB, LILO, etc.

Install & Configure Ubuntu to boot from an iSCSI disk

NOTE: This is Part 3 in a series on booting Ubuntu from an iSCSI disk. We have previously created an iSCSI disk and connected to it from the Ubuntu Live installation CD. View all of the sections here.

Many of the commands in this post were taken from http://etherboot.org/wiki/sanboot/ubuntu_iscsi.

If you’re following along with the series, you are now on the first screen of the Ubuntu installation wizard and the iSCSI disk has been connected to your system. You can now continue the installation as normal. When you get to the disk selection screen, the iSCSI drive should show up as a normal drive; partition it as you like.

NOTE: After the installation completes, and before you reboot, we have a few more steps to complete. DO NOT REBOOT!

At this point if you rebooted, the boot process would fail with a message indicating that you don’t have a hard drive. If you are not a good listener and decided to reboot anyway, all is not lost. Simply boot to any Ubuntu live disk and follow the instructions in the previous post to connect to the iSCSI drive.

We need to configure the initrd process to connect to the iSCSI disk before it does anything else. In order to do this, we’ll need to mount our newly installed system. So once again open a terminal. Assuming your root file system was installed on iSCSI device /dev/sda1 you would enter

root@client:~$ sudo su
root@client:~$ mkdir /mnt/newsystem
root@client:~$ mount /dev/sda1 /mnt/newsystem
root@client:~$ chroot /mnt/newsystem
root@client:~$ mount -t proc none /proc
root@client:~$ hostname -F /etc/hostname

Now that we have switched from the live system to the system installed on the iSCSI disk, we need to install some packages.

root@client:~$ apt-get install initramfs-tools open-iscsi sysv-rc-conf

Next, we’ll want to add iSCSI to the list of modules to include in the boot image.

root@client:~$ echo "iscsi" >> /etc/initramfs-tools/modules

Setup the initiator’s unique ID with the following command. You’re welcome to use a different name, however, it must follow the same naming convention as before and cannot be the same as the name of the iSCSI disk itself. If, for some reason, you decide to allow multiple clients to connect to the iSCSI disk, each client computer will need its own unique ID.

root@client:~$ echo "InitiatorName=iqn.2012-03.com.example:xbmc-client1" > /etc/iscsi/initiatorname.iscsi

Touch the indicator file, so that the module is included in the new initrd.

root@client:~$ touch /etc/iscsi/iscsi.initramfs

Finally, we’ll update initramfs.

root@client:~$ update-initramfs -u

There are still a couple of items to clean up before we are ready to reboot. If the networking systems try to renegotiate an IP address, our connection to the iSCSI device will be lost and bad things will happen. So let’s switch the network configuration to manual.

root@client:~$ nano /etc/networking/interfaces

Change this file to read:

auto eth0
iface eth0 inet manual

Lastly we need to update GRUB to pass the iSCSI parameters to the kernel at boot time.

root@client:~$ nano /etc/default/grub

Append the following to GRUB_CMDLINE_LINUX_DEFAULT, substituting the appropriate values.

ip=dhcp ISCSI_INITIATOR=iqn.2012-03.com.example:xbmc-client1 ISCSI_TARGET_NAME=iqn.2012-03.com.example:xbmc-server ISCSI_TARGET_IP=192.168.0.101 ISCSI_TARGET_PORT=3260 ISCSI_USERNAME=<username> ISCSI_PASSWORD=<password>

Update GRUB to lock in the changes.

root@client:~$ update-grub

Now Ubuntu’s boot process is configured to boot over iSCSI, but we still have to setup iPXE to get the boot sequence started.

Initiating a connection to an iSCSI disk in Ubuntu

NOTE: This is Part 2 in a series on booting Ubuntu from an iSCSI disk. In the previous section, we setup an iSCSI target (a.k.a. server) which is sharing a blank iSCSI disk on the network. View all of the sections here.

Once again, the bulk of this post was unabashedly “borrowed” from howtoforge.com

The instructions in this post will work for any existing Ubuntu client, but if you are following along with the series, this is the part where you’ll want to boot your client machine from the Ubuntu CD / USB drive. Once booted, but before we start the installation, we’ll need to connect to the iSCSI disk so that it is available to install to. In order to do this, we’ll need to install the iSCSI initiator software onto the live (read: temporary) system.

When the installation wizard starts, drop to a shell. In my case, I right-clicked on the background and one of the context menu items was Terminal Emulator – perfect.

Jumping right in: get root, install the iSCSI initiator software and start the daemon

sudo su
root@client:~$ apt-get install open-iscsi
root@client:~$ /etc/init.d/open-iscsi start

Note: To reiterate, we are in a shell on the live system, so even though we just installed the iSCSI initiator software, when we reboot, everything we do here will be nullified. That’s not a bad thing; it’s just important to understand the distinction between the live (temporary) and installed (persistent) systems. We are setting up a connection from the live system so that the installation wizard can install the persistent system to the iSCSI disk.

Using the admin tool, run a discovery command to see what the server has to offer (where 192.168.0.101 is your server’s IP address)

root@client:~$ iscsiadm -m discovery -t st -p 192.168.0.101

You should see a description of the target disk that looks like this:

192.168.0.101:3260,1 iqn.2012-03.com.example:xbmc-server

If you don’t get a response, you may need to adjust your firewall settings on the server to allow access to port 3260 and/or confirm the iSCSI server is setup correctly.

Before we can connect we need to set the authentication settings which are stored in /etc/iscsi/nodes/iqn.2012-03.com.example:xbmc-server/192.168.0.101,3260,1/default. We can edit this file directly, or use the iscsiadm tool instead. These commands can be skipped if you opted not to use authentication when you setup your target.

root@client:~$ iscsiadm -m node --targetname "iqn.2012-03.com.example:xbmc-server" --portal "192.168.0.101:3260" --op=update --name node.session.auth.authmethod --value=CHAP
root@client:~$ iscsiadm -m node --targetname "iqn.2012-03.com.example:xbmc-server" --portal "192.168.0.101:3260" --op=update --name node.session.auth.username --value=<username>
root@client:~$ iscsiadm -m node --targetname "iqn.2012-03.com.example:xbmc-server" --portal "192.168.0.101:3260" --op=update --name node.session.auth.password --value=<password>

Finally, we can connect to the disk using the following command

root@client:~$ iscsiadm -m node --targetname "iqn.2012-03.com.example:xbmc-server" --portal "192.168.0.101:3260" --login

and for future reference, you can disconnect by using this command

root@client:~$ iscsiadm -m node --targetname "iqn.2012-03.com.example:xbmc-server" --portal "192.168.0.101:3260" --logout

You can use fdisk to confirm that the disk is available

root@client:~$ fdisk -l

Now that we have connected to the iSCSI disk, we are ready to continue with the installation, but I’ll leave that for another post.

Setup an Ubuntu iSCSI target (server)

NOTE: This is Part 1 in a series on booting Ubuntu from an iSCSI disk. View all of the sections here.

The bulk of the instructions in this post were unabashedly “borrowed” from howtoforge.com.

The first thing we need in our quest to boot Ubuntu from an iSCSI disk is an iSCSI target disk. This is a real or virtual disk on your existing Ubuntu server that will be shared on the network. The Ubuntu Server OS is not actually required. Any desktop version of Ubuntu will work fine. However, I’ll continue to reference this computer as the server since it will be serving the disk to the client machine. Since I’m using XBMCbuntu Eden as my client OS, I chose to create a 20GB virtual disk in the /mnt/media folder on my server as follows:

sudo su
root@server:~$ mkdir /mnt/media/iscsi
root@server:~$ dd if=/dev/zero of=/mnt/media/iscsi/xbmc.img bs=1M count=20000

That last command will write 1 Megabyte worth of zeroes to xbmc.img 20,000 times resulting in a ~20GB image file. Now that we have a virtual disk we need to share it. To install the iSCSI target software type:

root@server:~$ apt-get install iscsitarget

The software is disabled by default, so we need to open the configuration file /etc/default/iscsitarget in your favorite text editor and set ISCSITARGET_ENABLE=true

root@server:~$ nano /etc/default/iscsitarget

Next we define our share. The letters iet stand for iSCSI Enterprise Target.

root@server:~$ nano /etc/iet/ietd.conf

Comment out everything in this file and add this stanza which defines how we will share our virtual disk. The standard naming convention for an iSCSI target is the letters “iqn” followed by the year and month the disk was created, followed by the domain name of the server in reverse order and lastly a unique identifier for this share. Substitute your own authentication values for <username> and <password> or remove them to allow anyone to connect to the disk.

Target iqn.2012-03.com.example:xbmc-server
        IncomingUser <username> <password>
        OutgoingUser
        Lun 0 Path=/mnt/media/iscsi/xbmc.img,Type=fileio
        Alias xbmc-server

Note: RFC 3720 requires <password> to be 12 characters long.

The share definition should be mostly self-explanatory:

1. The first line defines the name of the target. This is the name that our initiator will later use to find and connect to the disk.
2. The second line defines the user credentials for connecting to the disk. You can omit <username> and <password> if your middle name is Danger.
3. OutgoingUser defines the username and password to authenticate the target to the initiators during the discovery process, although we’ve shown that these credentials can also be omitted.
4. Line 4 sets up the image file we created above to be the source of our shared disk.
5. Lastly we provide an optional alias for the target disk.

By default the disk is shared with the entire network. If you’d like to lock that down, edit the following file

root@server:~$ nano /nano /etc/iet/initiators.allow

Comment out everything in this file and add this stanza which limits the share to your local network.

iqn.2012-03.com.example:xbmc-server 192.168.0.0/24

Lastly, start the service

root@server:~$ /etc/init.d/iscsitarget start

PXE Booting Ubuntu from an iSCSI drive

The Plan

This will serve as a guide to booting Ubuntu (XBMCbuntu in my case) from an iSCSI drive.

The Result

The boot process for my XBMCbuntu machine follows these steps:

1. The BIOS is configured to PXE boot.
2. The PXE boot process launches iPXE (formerly gPXE).
3. iPXE connects to the iSCSI drive and launches GRUB which is installed on the remote iSCSI disk.
4. GRUB launches a custom initramfs that has been compiled with the iSCSI module.
5. GRUB passes the iSCSI connection information to the kernel via command line options.
6. The kernel informs initramfs to initiate a connection to the iSCSI disk.
7. The kernel boots using the iSCSI disk as its root file system..

This is the resulting storage information screen in XBMCbuntu. For all intents and purposes, the OS thinks it has a local 20GB disk when in reality the disk is on a different machine and the system has been booted over the network.

The Definitions

• PXE or Pre-boot eXecution Environment is a method of booting computers via a network interface without any local hard drive. In simpler terms, the network card gets the files it needs to boot from a file server on the network rather than from a hard drive inside the computer.
• iPXE (formerly gPXE) is PXE on steroids; it allows booting from additional network protocols such as iSCSI.
• iSCSI is a protocol for accessing disks (virtual or physical) over a network at the block level. This is very different from other protocols such as SAMBA or NAS which do not provide block level access to shared disks.
• iSCSI initator is the diskless, client computer. It initiates the connection to the iSCSI target.
• iSCSI target is the server which shares the iSCSI disk on the network.
• Lastly, XBMCbuntu Eden is a flavor of ubuntu designed to be a media center, and incidentally will be the Ubuntu flavor that I will be installing on my client.

The Roadmap

It is assumed that you already have a machine running Ubuntu that will act as the iSCSI target aka the server.

• Part 1: Setup the Ubuntu iSCSI target (server) - In Part 1 we will setup a virtual disk drive that will be shared to the network over the iSCSI protocol.

• Part 2: Initiate an iSCSI Connection - In Part 2 we will move to the client computer and initiate a connection to the iSCSI disk we created in Part 1.

• Part 3: Install & Configure Ubuntu to boot from an iSCSI disk – In Part 3 we install the OS, build a new initramfs and configure GRUB to support booting from a remote iSCSI disk.

• Part 4: Install & Configure iPXE – In Part 4 we setup an (optional) PXE server and setup iPXE to connect to and boot the iSCSI disk.

• Part 5: Configuring dnsmasq to chainload PXE, iPXE and GRUB – connecting the dots to boot the diskless client system.