Because sometimes, you just want to do something crazy.
Tag Archive: OpenBSD
Let’s start with a note to self: when copying virtual machines, be sure to generate the MAC addresses again, otherwise you may end up with two virtual machines sharing the same MAC address. That explains why CARP wouldn’t run at the first try.
But what is CARP? It stands for Common Address Redundancy Protocol, and works the same as HRSP, VRRP and GLBP: it allows several routers to share a virtual IP which acts as the gateway for connected hosts. When one of the routers fails, another takes over the virtual IP so network connectivity for the hosts remains.
CARP has quite a history, which you can read in detail on Wikipedia. Because of it, CARP uses the same IP protocol as VRRP (112) and thus will show up as VRRP in Wireshark.
Configuration, with persistence between reboots, is similar to the interface configuration and bridging setup: CARP uses a special interface which, you guessed it, is created at boot if the file /etc/hostname.carp0 is found. However, I was unable to find the correct syntax in the file for OpenBSD 5.0, and the ones suggested in the manuals don’t work. However, just having the file already creates the interface, and everything with an exclamation mark in front of it inside a hostname file will be executed as a command, so the following line works:
- !ifconfig carp0 vhid number ip-address netmask subnetmask
- The vhid number is the CARP group. You can have more than one CARP group per interface, but for a given group, the configuration has to be the same on all devices.
- ip-address is the virtual IP that can be used as gateway for the hosts on the subnet.
- Optionally, you can add ‘pass password‘ in the command to secure the CARP packets with a password.
- Also optional, ‘advskew number‘ is a value between 0 and 254. The OpenBSD CARP with the lowest advskew value becomes the master.
Other options are possible, but these are the most important to get everything going. If things don’t work yet, it’s likely that pf is blocking the CARP packets. ‘pass in quick on em0 proto carp’ and ‘pass out quick on em0 proto carp’ solve this. Keep in mind all filtering still has to be done on the physical interface, filtering anything on ‘carp0’ will not take effect.
Finally, just like with the other gateway redundancy protocols, there’s a preempt option. When preempt is disabled, the first active OpenBSD will become master, even if other OpenBSD’s with a lower advskew value become active. When it’s enabled, the OpenBSD with the lowest advskew value will become master, whether the currently active OpenBSD has failed or not. The value can be manipulated in /etc/sysctl.conf, where net.inet.carp.preempt has to be set to ‘1’ (or just remove the ‘#’ if it’s already present but commented out).
Since I’ve covered enough for a complete setup, my next post will not be about OpenBSD anymore. Stay tuned!
The past weeks have been very informative for me and I learned a lot about many network-related things. As you may have noticed, I’ve been giving my attention to OpenBSD mostly, as I am determined to learn and control this system. It’s low cost allows for some home use.
More parts will follow, however, I’ve run into problems and my OpenBSD’s do not react as expected for the moment, and sometimes don’t react at all. I’m unsure what is causing it, but it appears to be related to the fact that I’m running all my OpenBSD’s in VMware Workstation 8. They don’t accept certain packets destined for them, even though they show up in tcpdump. I added firewall rules in pf and eventually even disabled pf entirely (the command: ‘pfctl -d’ for disabling and ‘pfctl -e’ for enabling), but it didn’t change anything.
I’ve also sold part of my home lab: two 2611’s and the 2900XL, as well as a console cable, a serial cable, and some Ethernet cables. Stuff I don’t really need anymore, but provide a decent CCNA lab to another motivated network engineer-to-be out there.
And finally, I’ve had the chance to do BGP troubleshooting, when it was discovered a local ISP didn’t provide routing for certain IP prefixes. A very important tool in this is a Looking Glass Server, which is a router that has an iBGP session with an ISP BGP router, on which you can query for the reachability of routes on the internet. Useful, even for single-homed connections, to see if there is an ISP problem. The Looking Glass I use are those of Level 3 Communications: http://lg.level3.net/.
That’s it for this week. For next week: either more OpenBSD, or wireless and EIGRP, depending on what works and what not. Greetings!
So far I’ve used OpenBSD as a layer 3 (routing) and layer 4 (firewalling) device, but it also has layer 2 functionality. For example, it’s possible to bridge between interfaces and use two, three, or more NICs as a logical switch.
Setting it up is quite easy:
- All interfaces part of the bridge have to be up, which can be done by making their respective /etc/hostname.int files and adding the word ‘up’ in each file.
- The bridge has to be defined with /etc/hostname.bridge0. Add one line per interface: ‘add interface‘, and the word ‘up’ again.
- There’s no IP needed on each port as it’s a layer 2 thing now. You can define an IP though.
- net.inet.ip.forwarding and net.inet6.ip6.forwarding do not have to be activated for this, as it’s not forwarding, but bridging.
You can now still use pf for filtering on the interfaces, and create a layer 2 transparent firewall. Keep in mind that if you’re used to hardware switching, this is all done in software now, and the NICs will be in promiscuous mode, so high throughput will require high processing power. According to ifconfig, spanning-tree seems to be active on the interfaces too (which should be!) but I was unable to test it.
Update 28/01/2012: spanning tree works after adding the lines ‘stp interface’ to /etc/hostname.bridge0, one for each interface.
Using one interface as a trunk link is possible already with simple bridging, as any frame will be passed on, but to let OpenBSD participate in the VLANs, an SVI can be defined:
- Create a file /etc/hostname.vlan, e.g. /etc/hostname.vlan5 . Add the IP address line, just like in a physical interface, but bind it to the trunk link with the ‘vlandev’ command: ‘inet ip-address subnetmask vlan vlan vlandev interface‘
- If you want to do inter-VLAN routing, net.inet.ip.forwarding and net.inet6.ip6.forwarding need to be set to 1. Otherwise the interface will work, but no routing will take place.
- Unfortunately, it only works on one physical interface, so it restricts the OpenBSD to a router-on-a-stick kind of configuration.
Now the OpenBSD can do inter-VLAN routing, and accepts 802.1q tagged frames. Note that the physical interface can still have an IP address which will be used for the untagged traffic on the interface.
I’ve already set up an IPv6 tunnel on three platforms: Vyatta, Cisco and Windows Server. This time, the same on OpenBSD. I’m not going to repeat myself, so for details about an IPv6 tunnel and how to get one, check the IPv6 tunnel article. I’ll be using the same example values again:
Local IPv6 subnet: 2001:0:0:1234::/64
Tunnel subnet: 2001:0:0:1235::/64, with ::2 on our side and ::1 on the other endpoint side.
IPv6 DNS: 2000::2000
Device IPv4 address: 192.168.0.10
Tunnel endpoint: 50.60.70.80
Gateway to ISP: 192.168.0.1
I assume routing and IPv4 is configured properly already, with IP’s on interfaces and a default route towards the internet. If not, you’ve missed part I. Before starting the IPv6 part, remember that you’ll be creating a tunnel over an existing IPv4 network, so make sure pf allows the tunnel. I’ve added the following rules in /etc/pf.conf:
pass out quick on em0 from 192.168.0.10 to 50.60.70.80
pass in quick on em0 from 50.60.70.80 to 192.168.0.10
You’ll need to pass both ipv6ip and icmp, but since it’s just one trusted IP address, I’m doing a general rule. Don’t forget to activate the rule with ‘pfctl -f /etc/pf.conf’!
Next, creating the tunnel interface. In OpenBSD this is a ‘gif’, generic interface. To make it persist between reboots, create a /etc/hostname.gif0 file, zero for the first tunnel interface. The following lines go in the file:
tunnel 192.168.0.10 50.60.70.80
!ifconfig gif0 inet6 alias 2001:0:0:1235::2 2001:0:0:1234::1 prefixlen 128
!route add -inet6 default 2001:0:0:1235::1
The internal IP 192.168.0.10 is automatically translated by my router, but this may not always be the case. If not, use your external IP. The prefix length in the second line is 128, which is advised in the tunnelbroker configuration sample, but I’m not sure why. It wouldn’t work with 64 though. Finally, the third line adds a default route into the tunnel.
At this point the tunnel is up and running, but from the OpenBSD only. The devices on the connected subnet are not aware an IPv6 router is present. For this, the OpenBSD will have to send router advertisements. First, configure an IPv6 address on the interface, by adding the following line to /etc/hostname.em1:
!ifconfig em1 inet6 alias 2000:0:0:1234::1 prefixlen 64
Next, do the actual advertisements using the rtadvd deamon. In /etc/rc.conf, find the ‘rtadvd_flags:NO’ and change the ‘NO’ to the interface(s) that need it enabled, e.g. em1. Then create the file /etc/rtadvd.conf’ and enter the following:
em1:\
:addr=”2000:0:0:1234::”:prefixlen#64:
This advertises the /64 prefix on the interface. A lot of other options are possible, such as the other-config-flag and managed-config-flag for DHCP options and a IPv6 DNS server, but I will not go into detail about that now. Finally, keep in mind icmp is used for router advertisements and neighbor discovery (the ARP replacement), so you’ll need to allow these. In /etc/pf.conf:
pass out quick on em1 inet6 proto icmp6
pass in quick on em1 inet6 proto icmp6
Finally, add some rules based on what you want to filter, e.g. a general rule blocking everything IPv6 inbound, and allowing outgoing connection of any kind (for now):
pass out quick on gif0 inet6 from 2000:0:0:1234::/64 to any
block in on gif0
After this, surfing to ipv6.google.com is possible from any computer in the local subnet.
In this third part of my series of OpenBSD posts, I’ll cover static routing and OSPF. Routing between connected interfaces is already covered in part I, but that doesn’t get you very far in a bigger network.
First static routes: these are very easy to configure, just type ‘route add subnet gateway‘, where subnet is the subnet or prefix, and gateway‘ the next-hop IP, e.g. ‘route add 10.0.0.0/8 192.168.168.2’ defines a static route for 10.0.0.0/8 to 192.168.168.2. Simple as that.
But just like with the other commands so far, this doesn’t persist after a reboot. To do this, the commands have to be automatically loaded at boot, which means they’ll have to be specified in a file that loads at boot time. Best candidates are /etc/rc.local and an interface config file, e.g. /etc/hostname.em0. Just add the command on a line, preceded by ‘!’, e.g. ‘!route add 10.0.0.0/8 192.168.168.2’, and save the file in vi.
OSPF takes OpenBSD to a whole new level, making it capable of supporting large network topologies. It has to be enabled on boot first by modifying the /etc/rc.conf file. After opening, find the ‘ospfd_flags=NO’ line and remove the ‘NO’ (don’t place anything else, just leave the line empty). After this, you can modify the OSPF behaviour in /etc/ospfd.conf. Below a simple sample configuration:
router-id 5.5.5.5
area 0.0.0.0 {
interface em0
interface em1 {
passive
}
}
The router-id speaks for itself. Area 0.0.0.0 means area 0, in which interfaces em0 and em1 are participating. Interface em1 is defined as passive. Basically everything that’s possible on a Cisco router is possible here too, except NSSA which is Cisco-only.
In my case, this still doesn’t work. Why? Because of the pf settings: I haven’t allowed the interface IP to send traffic to 224.0.0.5. For this, I add the rule ‘pass out quick on em0 from 192.168.168.5 to 224.0.0.0/24’ to /etc/pf.conf. After this OpenBSD starts sending out OSPF Hello’s, so if you’re ever having trouble with this, check pf.
Since I’m working multi-vendor already, I’m adding an extra here and boot up a Vyatta 6.3 and Cisco 3560, configure OSPF, and connect them to the same subnet. This is the result:
6.6.6.6 is the Vyatta, 1.1.1.1 the Cisco. Through some reboots the Cisco became the only active device at a certain moment, thus becoming DR. Both the Vyatta and the Cisco show an OSPF route for the em1 interface of OpenBSD, so everything works.
In my previous post I described how to set up OpenBSD and enable simple routing between interfaces. This time an explanation about one of the core features: pf, or packet filter.
The pf service allows for stateful firewalling and NAT translations when routing. Starting where we took off in part I, we have an OpenBSD 5.0 with routing enabled and interfaces configured. pf is one of the few things to be active by default, so it’s already running now, but it doesn’t do anything as no rules have been defined yet. Defining rules has to be done in /etc/pf.conf . How? Again, by using a text editor like vi. Yes, that’s ‘vi /etc/pf.conf’ on the command line. There are already some basic rules defined in pf.conf, most of them commented out as examples.
A filtering rule consists of several keywords. A complete guide is available as a manual page, but I’ll list the basics here:
- Most used are pass and block. The first permits a packet based on parameters defined further in the rule, the second drops the packet.
- Next is in or out. Either filter incoming or outgoing on an interface. Incoming takes place before a route look-up so filtering mostly on incoming will take less CPU.
- This one is optional: quick. Without it, pf will check all rules and treat a packet according to the last matching rule. However, if a rule contains the word quick and the packet matches, any following rules are no longer checked. Using this keyword makes the filtering behave more like a Cisco ACL (top-to-bottom), and will take less CPU as not all rules have to be examined every time.
- on, followed by an interface, e.g. em0. It defines on which interface the filtering takes place.
- proto, followed by a protocol, mostly tcp, udp or icmp. It’s an optional keyword: if it’s missing, filtering will be done with layer 3 information (IPv4, IPv6) only.
- from defines where from. It can be any, an ip, or a subnet, e.g. 10.0.1.0/24.
- port is optional after this and defines the source port(s). It uses operators, e.g. port = 2000 is port 2000, port < 5000 is any port below 5000, and so on. Since it’s a source port and most applications use dynamic source ports, this isn’t used that much.
- to defines the destination host or subnet. Same parameters as from.
- A second port, again optional, after to. This is the destination port. Since these are usually well-known ports, it’s often used.
Many more options and keywords are possible, but knowing the above is sufficient for decent stateful firewalling.
The second part is address translation, or NAT. A basic setup is as following:
- match is the first keyword. You can also use pass, but it will pass everything unless you define certain ports in the rule also.
- Next are in or out, with out being the most used (and easiest to troubleshoot too, in my opinion).
- Then: on for the interface, from and to. proto and port are optional.
- Last: nat-to, followed by an (external) IP. It defines what the host or subnet defined in from will NAT to.
We can try this in an example. Let’s assume the OpenBSD has interface em0 connected to an ISP (IP 1.2.3.4) and em1 to an internal network 192.168.168.0/24. This shows what can be done with it:
match out quick on em0 from 192.168.168.0/24 to any nat-to 1.2.3.4
pass out quick on em0 proto tcp from 1.2.3.4 to any port = 80
pass out quick on em0 proto udp from 1.2.3.4 to 8.8.8.8 any port = 53
block out on em0
The above will NAT any internal address from em1 to the external IP. Everything will be blocked (the last rule), except TCP traffic to port 80 (which is http traffic) and UDP port 53 to IP 8.8.8.8 (which are DNS requests to the IP of Google’s DNS).
The above works, but it’s a very incomplete example of course: no https (port 443) is possible for example, as are pings and the ability to choose another DNS server in case of a DNS problem. But it demonstrates the point. Don’t forget to do ‘:wq’ to save your changes!
Now just one more thing: these rules aren’t activated by default: you’ll need ‘pfctl -nf /etc/pf.conf’ and ‘pfctl -f /etc/pf.conf’. The first commands checks if the rules don’t contain any syntax errors, which you should always check, and the second command loads the rules in the current configuration.
Time for something slightly different: in the upcoming weeks, I’m going to post several articles about OpenBSD. OpenBSD is a command line Unix operating system, which by default comes with pf (short for packet filter) and OSPF and BGP daemons. A daemon, in Unix terminology, is what a service is for Microsoft: a piece of software accepting and making connections for something. And pf is stateful firewall software, capable of filtering on layer 3 and layer 4 of the OSI model, as well as performing NAT. Newer versions even support IPv6 routing, firewalling and OSPFv3.
I’ll be using the newest version, OpenBSD 5.0. Given that it uses less than 1% CPU and less than 40 MB of RAM in idle state, it makes a very lightweight platform. It’s also not running many services that can be exploited. It can be downloaded for free from the OpenBSD website. Installation is so straightforward I’m not even going to cover it here, and going with default options will give you a working system. Using it in a VM is no different compared to a bare-metal installation.
After installing, rebooting, and logging in, you’ll be in the command line. If you didn’t log in as root, you can gain root privileges with the ‘su -‘ command. The first thing to do is to enable routing, as OpenBSD will not do this by default. But first some basics about the OpenBSD environment. ‘sysctl’ is an interface to view and modify system parameters. Simply typing ‘sysctl’ in the command line will give all system variables, which is a long list. To filter long output in any command in OpenBSD, you can use ‘grep’, which has to be piped after the command, e.g. ‘sysctl | grep net.inet.ip.forwarding’ will show only the net.inet.ip.forwarding variable, which is currently set to zero. Note that you can ‘grep’ as specific as you want, e.g. ‘sysctl | grep net.inet.ip’ will show all IP variables containing ‘net.inet.ip’.
Changing a variable can be done easily by typing ‘sysctl variable=value‘, e.g. ‘sysctl net.inet.ip.forwarding=1’ activates IP routing, but only until the next reboot. To make a system variable permanent, you’ll have to change it in the file /etc/sysctl.conf (the /etc directory contains most configuration files). To change this file, you can use the build-in text editor vi: ‘vi /etc/sysctl.conf’. Note that vi works very different from a Windows-based text editor and you can’t immediately start typing. A full manual can be checked here. Normally,’ net.inet.ip.forwarding=1′ should already be visible in the file but commented out with a #, which means it is ignored. Go to the # and press ‘x’ to delete it. The same can be done for ‘net.inet6.ip6.forwarding=1’, which enables IPv6 routing. Saving the change can be done by typing ‘:wq’, where ‘:’ means menu or file, ‘w’ is write, and ‘q’ is quit. To quit without saving, type ‘:q!’ and you’re back to the command line.
Next, configuring interfaces. Type ‘ifconfig’ to view the interfaces. The first physical interface is ’em0′, and if more are present, those are ’em1′, em2, and so on. To configure an IP address on an interface, use ‘ifconfig interface ip netmask subnetmask‘, e.g. ‘ifconfig em0 172.16.1.1 netmask 255.255.255.0’. To shutdown and restart the interface (if needed), use ‘ifconfig em0 down’ and ‘ifconfig em0 up’, respectively. But just like with sysctl, this is only until the next reboot. A permanent configuration is done by creating a filed name ‘hostname.interface in /etc. Strangely enough, it’s literally ‘hostname’, no matter the real hostname of your system. So the first interface is defined in /etc/hostname.em0. To create the file, just try to open it in vi and it should create automatically. Add the line ‘inet ip-address subnetmask‘ and you’re done. Optionally, you can add the line ‘!route add default gateway-ip‘ in one of the files, which will define the default gateway of your system.
Most of my research was done using Google, so if you run into trouble, search and you’ll find. Also, OpenBSD contains manual pages which can be consulted with the ‘man’ command, e.g. ‘man vi’ and ‘man sysctl’. To quit the manual, typ ‘:q’.
That concludes the basic intro and enabling routing for OpenBSD. In upcoming blog posts, I’ll cover the following:
- Firewalling and NAT using pf
- OSPF using the ospfd
- IPv6 functionality
- NTP
- Perhaps CDP and LLDP support
Stay tuned!
Reggle 