User Tools

Site Tools


netbsd_stratodedi

Install NetBSD 6.0.1 amd64 on a STRATO PowerServer LR v5.4

Summary

This document describes a possible strategy to setup NetBSD 6.0.1 amd64 on a STRATO PowerServer LR v5.4.

To pull this off, we will use the serial console and the network-bootable rescue system provided by STRATO. We will also have to build a custom kernel first. We will then download and write a prepared bootable intermediate disk image to our machine, which enables us to boot a ramdisk. A full guide on how to build such an image is included in this document. The basic installation is done using IPv6 and FTP as installation source from the fr.netbsd.org mirror. Networking is quite unstable in the INSTALL.STRATO kernel, so be quick and only do a basic install. Now, before you laugh, the resulting system is production-stable and currently used to serve a pretty busy web radio station. This howto was done primarily for myself as a cheat sheet.

This whole document is work-in-progess. Nevertheless, a usable disk image and other related files can be found contained in this tarball.

The fr mirror of NetBSD was chosen because:

  1. All german mirrors are pretty slow (no, i cant afford to provide another one)
  2. The french mirror is the next closest after the german mirrors.

TODO

  1. Custom kernel (done but undocumented yet)
  2. Linux binary compatibility (done but not documented yet)
  3. Disk encryption
  4. Disk quotas (for documentation see # cd /usr/share/doc/smm/04.quotas; make) (done but not documented yet)
  5. IPv6 “routing” (rather: terminating an IPv6 subnet on your server and use a couple of the available IPs)
  6. faithd (v4→v6)
  7. firewalling (done but not documented yet)

Prerequisites

At the moment, this process only works via IPv6. Thus you have to enable it for your server. Basic IPv6 support is enough. The customer service panel has howtos on the configuration.

From the rescue system or your old installation, collect _at least_ the following info:

  1. 'dmesg' output
  2. 'ifconfig -a' output
  3. 'netstat -nr' output
  4. Serial console setup (usually 57600 baud, com0)

To sum it up, you'll need to write down the network config, routing info and serial console config. Also, BLAH BLAH backup BLAH.

Prepare intermediate disk image

Get NetBSD 6.0.1 sources via FTP and unpack them to /usr/src:
# cd /tmp
# ftp -i ftp://ftp.fr.netbsd.org/pub/NetBSD/NetBSD-6.0.1/source/sets/
> mget *.tgz
> quit
# cd /
# for i in /tmp/*.tgz; do tar zxhepf $i; done
# rm /tmp/*.tgz

You might as well delete the xsrc.tgz disk set, we won't need it.

Set console port and speed in /usr/src/sys/arch/amd64/conf/GENERIC:
options CONSDEVNAME=“\”com\“”,CONADDR=0x2f8,CONSPEED=57600

Build a release and copy ramdisk.fs, the INSTALL kernel and other necessary files to ~/netbsd-strato:
# cd /usr/src
# ./build.sh -u tools
# ./build.sh -u release
# mkdir ~/netbsd-strato
# cp /usr/src/distrib/amd64/ramdisks/ramdisk/obj/ramdisk.fs ~/netbsd-strato
# cp /usr/src/obj/sys/arch/amd64/compile/INSTALL/netbsd ~/netbsd-strato
# cp /usr/mdec/boot ~/netbsd-strato

Create a small boot.cfg file that boots without ACPI (-2) and tells the kernel that we have a ramdisk (-m):
timeout=-1
consdev=com0
banner=
banner=NetBSD 6.0.1 amd64 setup
banner=STRATO PowerServer LR (v5.4)
banner=
menu=Setup:boot netbsd -2m
menu=Boot prompt:prompt

See the BOOT_FLAG(9) manpage for details on boot flags.

Attach ramdisk.fs to INSTALL kernel, create, prepare, fill and compress the 10MB intermediate disk image. Copy away to server directly reachable from the internet (as in not behind a nat gateway):
# mdsetimage -v -s ~/netbsd-strato/netbsd ~/netbsd-strato/ramdisk.fs
# gzip netbsd
# dd if=/dev/zero of=~/netbsd-strato/disk.img bs=4k count=2560
# vnconfig -c vnd0 ~/netbsd-strato/disk.img
# disklabel -e -I vnd0
# newfs /dev/vnd0a
# installboot -v -m amd64 -o timeout=10,console=com0,speed=57600 -t ffs /dev/rvnd0a /usr/mdec/bootxx_ffsv1
# mount /dev/vnd0a /mnt
# cp boot* /mnt
# cp netbsd.gz /mnt/netbsd
# umount /mnt
# vnconfig -u vnd0
# gzip -9 ~/netbsd-strato/disk.img
# scp ~/netbsd-strato/disk.img.gz user@server.tld:/tmp

Installation

1) Set server to rescue boot mode, write down login credentials. Then reboot the server into rescue mode. The boot mode can be configured in the customer service panel.

2) Get the prepared intermediate disk image and write it to the first hard disk using dd. Then overwrite the first 10MB or so of the 2nd disk with zeroes just to be sure.
# scp user@server.tld:/tmp/disk.img.gz /tmp
# gunzip /tmp/disk.img.gz
# dd if=/tmp/disk.img of=/dev/sda bs=4k
# dd if=/dev/zero of=/dev/sdb bs=4k

3) Reset server to normal boot mode and reboot. The boot mode can be configured in the customer service panel.

4) Exit sysinst, configure the primary network interface with your primary IPv6 address:
# ifconfig wm0 inet6 dead:beef:421b:2d00:98f7:78cf:d1b0:7ae1 prefixlen 128 alias
# route add -inet6 default fe80::1%wm0

5) Exit the shell using CTRL+d or type 'exit' to return to sysinst.

6) Disable IPv4 networking by not specifying anything whereever possible, but say yes to IPv6 autoconfiguration.

7) Install NetBSD the usual way using ftp.fr.netbsd.org as installation source to the first hard disk. Only select the 'base' and 'etc' disk sets, otherwise we possibly won't be done downloading before IPv6 stops working.

8) Mount the first hard disk, edit /boot.cfg, /etc/rc.conf, /etc/ifconfig.wm0, /etc/resolv.conf, create /root/.ssh/authorized_keys, edit /etc/ssh/sshd_config:
# mount /dev/wd0a /mnt
# mkdir /mnt/root/.ssh /mnt2/root/.ssh
# chmod 0700 /mnt/root/.ssh /mnt2/root/.ssh

/root/.ssh/authorized_keys:
# chmod 0600 /mnt/root/.ssh/authorized_keys

/etc/ssh/sshd_config:
PermitRootLogin without-password

/etc/rc.conf:
hostname=netbsd
ip6mode=host
sshd=YES
ntpd=NO

/etc/ifconfig.wm0:
up

# IPv4 config
inet X.Y.132.210 netmask 0xffffffff
# 2nd IPv4 address
inet alias X.Y.247.235 netmask 0xffffffff
!route add -host X.Y.128.1 -link wm0 -iface -static -expire 1
!route add default X.Y.128.1 -ifp wm0

# IPv6 config
inet6 dead:beef:421b:2d00:98f7:78cf:d1b0:7ae1 prefixlen 128 alias
!route add -inet6 default fe80::1%wm0

/etc/resolv.conf (OpenDNS IPv4&IPv6):
# OpenDNS IPv4 & IPv6 nameservers
nameserver 208.67.222.222
nameserver 208.67.220.220
nameserver 2620:0:ccc::2
nameserver 2620:0:ccd::2

/boot.cfg:
clear=1
consdev=com0
timeout=-1
banner=
banner=NetBSD 6.0.1 amd64
banner=
menu=Boot default (Disable ACPI):boot netbsd -2
menu=Boot default (Disable ACPI & SMP):boot netbsd -12
menu=Boot default:boot netbsd
menu=Boot default single user:boot netbsd -s
menu=Boot strato (Disable ACPI):boot netbsd.strato -2
menu=Boot strato (Disable ACPI & SMP):boot netbsd.strato -12
menu=Boot strato:boot netbsd.strato
menu=Boot strato single user:boot netbsd.strato -s
menu=Drop to boot prompt:prompt

See the BOOT_FLAG(9) manpage for details on boot flags.

9) Reboot. The server should now boot into NetBSD 6.0.1 amd64 from the first hard disk.

Configuration

Custom optimized kernel

This tarball contains a sample kernel config file which can be used as a starting point in building a custom kernel on your home machine, in a virtual machine, or anywhere else possible. The Cheatsheet describes a method to transfer files to the server. The NetBSD Guide has a wonderful howto on building and optimizing the NetBSD kernel.

RAIDframe: RAID1

See the list of steps below and use the NetBSD Guide as a reference.

1) Compile information about intented target disk layout (fdisk, disklabel)
2) Init raid0 with wd1 and a dummy component
3) Create fdisk partitions and disklabel on wd1 with info from 1), create filesystems.
4) Remember to do it for each slice. Mount all slices. Sync running environment into raid0. Edit /mnt/etc/fstab (wd0 → raid0), edit /mnt/etc/rc.conf (swapoff=YES), make wd1 bootable
5) Boot from raid0 (raid0a:netbsd -2), insert wd0 into raid0, fail component0, replace with wd0, make wd0 bootable
6) Wait until rebuild is finished (raidctl -S raid0)
7) Done.

Proper testing of the boot sequence for both disks requires the precise elimination of the wd0 bootsector to force the server to boot from wd1 (and vice versa).

Linux binary compatibility

TODO. This is mostly useless without reliable networking but should work as specified in the NetBSD Guide.

Disk encryption

TODO. See the NetBSD Guide for details.

Cheatsheet

Copying stuff onto the server

As long as IPv4 does not work at all and IPv6 stops working after a short time, copying stuff on the server is a bit of a pain. Use this strategy to copy disk sets, kernel images, source sets and so on on to the server. Think twice while creating the tarball to write to the disk as the whole booting in and out of the rescue system takes some time.

1) Set server to rescue boot mode and reboot. The boot mode can be configured in the customer service panel.
2) Download the wanted files to /tmp.
3) Create tarball from downloaded files in /tmp.
4) Write tarball to 2nd hard disk using dd bs=4k, write down number of records written to disk.
5) Set server to normal boot mode and reboot. The boot mode can be configured in the customer service panel.
6) Extract tarball from disk using dd bs=4k and the number of records from step 4 as an argument to count=.
7) Untar tarball to suitable location.

The opposite way is done by performing step #4 in NetBSD and step #6 in the rescue system and exchanging steps #1 and #5.

Conclusion

Optimizations

  • Include all binary and source disk sets in the intermediate disk image. This could be done by increasing the ramdisk buffer in the STRATO kernel and attaching a ramdisk.fs containing the necessary files. The amd64 release directory has about 300MB. Considering that most home internet connections have a pretty limited outbound bandwidth, this might suck.
  • Include the intermediate disk image bootkernel in ramdisk.fs. This also requires increasing the ramdisk buffer size. The bootkernel has about 10 to 15MB. The machine description file for the STRATO PowerServer LR v5.4 is included in this tarball.
netbsd_stratodedi.txt · Last modified: 2013/04/17 16:10 by flo