IS-IS part II: areas and backbone.

Given the basics covered in part I, IS-IS configuration isn’t that hard. It already clearly shows some differences with OSPF, but it’s when using multiple areas that there is a clear distinction in logic.

First a small recap of OSPF areas: you have a backbone area, area 0, to which all other areas must connect. A router can be in multiple areas, an interface can be in only one area for a given OSPF process. Routes between areas are known by default, but setting an area to stub can change this to just a default route.

IS-IS is different: as you may have guessed by the ‘net’ command of part I, a router can only be part of one area. Area borders are between routers. An area is made up of routers with level 1 neighborships. A router with a level 2 neighborship towards another router is considered a backbone router. Since level 2 neighborships can be between routers in different areas (the second part of ‘net’ command can differ), these routers connect areas.

The moment a router has a level 2 neighborship and becomes a backbone router, it will automatically propagate a default route towards its level 1 neighbors. This gets flooded throughout the area. To reach another area, packets will be sent automatically towards the nearest backbone router. The Backbone router has a second topology table for level 2 that lists information of all subnets in all areas (which requires more memory). The packet will then be transported over the backbone to the appropriate area. For this reason, the backbone must be continuous: otherwise there would be multiple islands of routers propagating default routes.

From that point of view, the level 2 backbone becomes an overlay on top of the areas that connects everything: an extra ‘level’, likely the reason for the terminology. While this design works and is very scalable it may introduce suboptimal routing. Inter-area traffic will go to the nearest backbone router, but there may be other backbone routers in the area that can route the packets to the destination in a better way. For example, in the above image, the bottom router in the purple middle area may decide to follow the default route to the left backbone router for a packet destined for the right blue area.

Configuration is still straightforward:

Router(config)#interface GigabitEthernet0/1
Router(config-int)#ip address 10.0.2.5 255.255.255.252
Router(config-int)#ip router isis
Router(config-int)#isis circuit-type level-1
Router(config-int)#exit
Router(config)#interface GigabitEthernet0/2
Router(config-int)#ip address 10.0.3.1 255.255.255.252
Router(config-int)#ip router isis
Router(config-int)#isis circuit-type level-2-only
Router(config-int)#exit
Router(config)#interface GigabitEthernet0/3
Router(config-int)#ip address 10.0.2.9 255.255.255.252
Router(config-int)#ip router isis
Router(config-int)#
Router(config-int)#exit
Router(config)#router isis
Router(config-router)#log-adjacency-changes
Router(config-router)#net 49.0001.0000.0000.0008.00

This example configures a router for a level 1 neighborship on Gi0/1 (inside the area), a level 2 neighborship on Gi0/2 (between areas) and a level 1 & 2 neighborship on Gi0/3 (inside the area, but still backbone). Note the missing ‘is-type’ command in the routing process, which makes the router default to both a level 1 and level 2 router. A router in another area has a different area number in the net command:

Router(config)#interface GigabitEthernet0/2
Router(config-int)#ip address 10.0.3.2 255.255.255.252
Router(config-int)#ip router isis
Router(config-int)#isis circuit-type level-2-only
Router(config-int)#exit
Router(config)#router isis
Router(config-router)#log-adjacency-changes
Router(config-router)#net 49.0002.0000.0000.0009.00

Note that an IS-IS router is not required to have a level 1 neighborship. It is possible to have a ‘pure’ backbone router with only level 2 neighborships, which makes the router only use one topology table again, just like a level 1-only router.

The topology tables for both levels can be checked with show isis topology l1 and show isis topology l2. Same for the database, just replace the word ‘topology’ with ‘database’. The show clns is-neighbors and show isis neighbors commands both show all IS-IS neighbors and the level of the neighborship.

Advertisement

IS-IS part I: basics and single-level configuration.

IS-IS, or Intermediate System to Intermediate System. Just like OSPF, it’s a link-state routing protocol. This article took me quite a bit of research, and things were confusing for me at first because I kept looking at it from an OSPF point of view. Now that I’ve cleared that up for myself, I’ll do my best to explain it here for people knowing OSPF but not IS-IS (which, I assume, will be the majority of readers here).

First some explanation about why one would want to use IS-IS in the first place. After all, both are link-state routing protocols and OSPF is much more familiar to most. However, there are a few key differences in design of the protocols. But the most important reason to choose IS-IS over OSPF is scalability. IS-IS scales to larger topologies compared to OSPF using the same resources. A general recommendation for the number of OSPF routers in an area is between 70 and 100 maximum, while IS-IS will do 150 routers in an area (of course, the number of uplinks, routes and type of routers will influence this number). The difference in multi-area design can also make IS-IS more suitable for some topologies (which I will explain in part II later on).

This part will focus on a single area and basic configuration. It is useful to know some historical facts which explain the difference in commands compared to OSPF.

• Since IS-IS wasn’t designed with IP in mind but CLNS, it works directly on layer 2 with no IP headers. It uses flexible TLV (Type-Length-Value) fields in the PDUs it exchanges which makes it suitable for carrying routing information of just about any protocol. This is why it’s also used for IPv6 and even TRILL and FabricPath (which is actually nothing more than exchanging the location of MAC addresses by routing protocol).
• IS-IS has a concept of areas but refers to it as ‘levels’. On a Cisco router the IS-IS routing protocol will try to form neighborships for both level 1 and level 2 by default. When using just one area, it’s best to configure the routing protocol to form neighborship of level 1 only (again, multi-area will be covered in part II).
• A Network Entitity Title (NET) is used to identify a router. It is made up of four parts: the first byte is an Authority and Format Identifier (AFI),  next two bytes that define the area, followed by six bytes that act as a unique identifier (much like an OSPF router-id) and one byte for n-selector (NSEL). This NSEL is always set to zero for IS-IS for IP (non-zero values are used for actual data transport over CLNS, which likely isn’t used anywhere anymore). The AFI must be officially registered but 49 can be used for internal addressing.
• As a consequence, the first six bytes (AFI and area ID) have to be the same for all IS-IS routers in an area, and the following six bytes have to be unique for each IS-IS router in an area.
• For the unique ID part, several methods exist: you can use the system base MAC address, map an IP address to it, or simply start counting from 1 and up.

Given all the above, the basic IS-IS routing process can be configured as following:

Router(config)#router isis
Router(config-router)#log-adjacency-changes
Router(config-router)#is-type level-1
Router(config-router)#net 49.0001.0000.0000.0017.00

Unlike the other routing protocols, logging of adjacencies is not on by default on a Cisco router.

Now that the process is configured, interfaces must be added to it. That’s right, interfaces, no ‘network’ command to define subnets. This can be done in two ways:

• Configuring an IP address on an interface, followed by the ‘ip router isis’ command will make the interface participate.
• Configuring an IP address on an interface and defining that interface as passive in the router process will make IS-IS announce the subnet on the attached interface but not form any neighborships on it. The ‘ip router isis’ command is not required.

Router(config)#interface GigabitEthernet0/1
Router(config-int)#ip address 10.0.2.1 255.255.255.252
Router(config-int)#ip router isis
Router(config-int)#exit
Router(config)#interface Loopback0
Router(config-int)#ip address 10.0.10.14 255.255.255.255
Router(config-int)#exit
Router(config)#router isis
Router(config-router)#passive-interface Loopback0

And that’s it. Configure this on two adjacent routers and an IS-IS neighborship will form. You can check this using ‘show clns neighbors’ and ‘show isis neighbors’.

In upcoming parts, I’ll explain multi-area design and configuration and fine tuning of the default parameters. And for those interested, I’ve uploaded a capture of the IS-IS neighborship forming on Cloudshark.

Private VLANs: where to deploy them.

On this blog I’ve often covered Private VLANs: how to configure them, work around them and deploy them in a larger network. Yet it’s rarely that you see an actual Private VLAN in a design. Part of the problem is covered in the article about deployment over multiple switches: you can’t connect a trunked device such as a firewall to it. Although the Nexus 7000 provides a solution, that doesn’t make it much easier (or cheaper).

Another important reason is that few are willing to take the risk to deploy a VLAN where hosts cannot communicate with each other, as this is usually the reason hosts are put in the same VLAN in the first place. There’s the hesitation because it would introduce complexity or limit scalability, as new servers later on may need to communicate in the same subnet after all.

So where would it be beneficial and with low risk to use a Private VLAN? Actually quite a few places.

E-commerce

Say you have an internet-facing business with e-commerce websites where anyone can log in, create an account, or do a purchase. A compromised e-commerce server in the DMZ means immediate access to the entire DMZ VLAN. This VLAN has the highest chance of being compromised from the internet, yet the servers in it rarely need to speak with each other. In fact, if properly designed, they will all connect to backend application and/or database servers that on their turn communicate with each other. This way the e-commerce data is synchronised without the DMZ servers requiring a connection to each other.

Stepping Stones

Some environments have a VLAN with Stepping Stone servers where users can log on to with pre-installed tools to access confidential resources. Access from one Stepping Stone server to another is not needed here. Sometimes it’s even not desired as there may be a Stepping Stone per application, environment or third-party.

Out-of-Band
A modern rackserver has an out-of-band port to a dedicated chip in the server that can power off and on the server, and even install the OS remotely. For example, HP iLO. Typical here is that the out-of-band port never initiates connections but only receives connections for management, usually though the default gateway. This makes for a good Private VLAN deployment without issues.

Backup

Similar to out-of-band, some environments use a dedicated network card on all servers for backup. This introduces a security issue as it’s possible for two servers in different VLANs to communicate without a firewall in between. Again a Private VLAN can counter this. Somewhat unusual in the design is that it’s best to put the servers taking the backup in the promiscuous VLAN, so they can communicate with all servers and the backup VLAN default gateway, and put the default gateway in an isolated VLAN, preventing any other server from using it.

Campus – Wired guests
Similar to the Stepping Stones: guests can access the network through a firewall (the default gateway) but don’t need to access each others computers.

Campus – Wireless APs
In a WLAN deployment with a central controller (WLC), all the Lightweight APs do is connect to the controller using the subnet default gateway. Any other services such as DHCP and DNS will be through this default gateway as well.

Campus – Utilities
Utilities such as printers, camera’s, badge readers,… will likely only need the default gateway and not each other.

Where not to use PVLANs
This should give some nice examples already. But for last, a couple of places where not to use Private VLANs:

• Routing VLANs: unless you want to troubleshoot neighborships not coming up.
• VLANs with any kind of cluster in it: still doable with community VLANs for the cluster synchronisation, but usually better off in their own VLAN.
• User VLANs, VOIP VLANs and the like: VOIP and videoconferencing may set up point-to-point streams.
• Database server VLANs: not really clusters but they will often require access to each other.

Advantages of MPLS: an example.

While MPLS is already explained on this blog, I often still get questions regarding the advantages over normal routing. A clear example I’ve also already discussed, but besides VRF awareness and routing of overlapping IP ranges, there’s also the advantage of reduced resources required (and thus scalability).

Given the above design: two routers connecting towards ISPs using eBGP sessions. These in turn connect to two enterprise routers, and those two enterprise routers connect towards two backend routers closer to (or in) the network core. All routers run a dynamic routing protocol (e.g. OSPF) and see each other and their loopbacks. However, the two middle routers in the design don’t have the resources to run a full BGP table so the WAN edge routers have iBGP sessions with the backend routers near the network core.

If you configure this as described and don’t add any additional configuration, this design will not work. The iBGP sessions will come up and exchange routes, but those routes will list the WAN edge router as the next hop. Since this next hop is not on a directly connected subnet to the backend routers, the received routes will not be installed in the routing table. The enterprise routers would not have any idea what to do with the packets anyway.

Update January 17th, 2014: the real reason a route will not be installed in the routing table is the iBGP synchronisation feature, which requires the IGP to have learned the BGP routes through redistribution before using the route. Still, synchronisation can be turned off and the two enterprise routers would drop the packets they receive.

There are a few workarounds to make this work:

• Just propagating a default route of course, but since the WAN edge routers are not directly connected to each other and do not have an iBGP session, this makes the eBGP sessions useless. Some flows will go through one router, some through the other. This is not related to the best AS path, but to the internal (OSPF) routing.
• Tunneling over the middle enterprise routers, e.g. GRE tunnels from the WAN edge routers towards the backend routers. Will work but requires multiple tunnels with little scalability and more complex troubleshooting.
• Replacing the middle enterprise routers by switches so it becomes layer 2 and the WAN edge and backend routers have a directly connected subnet. Again this will work but requires design changes and introduces an extra level of troubleshooting (spanning tree).

So what if MPLS is added to the mix? By adding MPLS to these 6 routers (‘mpls ip’ on the interfaces and you’re set), LDP sessions will form… After which the backend routers will install all BGP routes in their routing tables!

The reason? LDP will advertise a label for each prefix in the internal network (check with ‘show mpls ldp bindings’) and a label will be learned for the interfaces (loopback) of the WAN edge routers… After which the backend routers know they just have to send the packets towards the enterprise routers with the corresponding MPLS label.

And the enterprise routers? They have MPLS enabled on all interfaces and no longer use the routing table or FIB (Forwarding Information Base) for forwarding, but the LFIB (Label Forwarding Information Base). Since all packets received from the backend routers have a label which corresponds to the loopback of one of the WAN edge routers, they will forward the packet based on the label.

Result: the middle enterprise routers do not need to learn any external routes. This design is scalable and flexible, just adding a new router and configuring OSPF and MPLS will make it work. Since a full BGP table these days is well over 450,000 routes, the enterprise routers do not need to check a huge routing table for each packet which decreases resource usage (memory, CPU) and decreases latency and jitter.

Introducing a backdoor on a Cisco router.

Yes, I’m riding the ‘OMG-NSA!’ wave, but it’s proven to be interesting. Eventually one starts pondering about it and even trying some stuff in a lab. Hereby the results: I’ve managed to introduce a backdoor in a Cisco router so I can log in remotely using my own username and non-standard port. Granted, it’s far from perfect: it’s detectable and will be negated if you use RADIUS/TACACS+. But if you’re not paying attention it can go unnoticed for a long time. And a mayor issue for real life implementation: it requires privileged EXEC access to do it in the first place (which is why I’m publishing this: if someone untrusted has privileged EXEC access, you have bigger problems on your hands).

The compromised system

The router which I tested is a Cisco 2800 series, IOS 15.1(2)GC. Nothing special here. The router is managed by SSH, a local user database and uses an ACL for the management plane.

The goal

Accidentally getting the password and gaining access is not a backdoor. I want to log in using my own private username and password, use a non-standard port for SSH access, and bypass the ACL for the management plane.

The Setup part 1 – Backdoor configuration

How it’s done: two steps. First, just plainly configure the needed commands.

• The username is configured.
• The non-standard high port is configured using a rotary group.
• The rotary group is added to the VTY (SSH) lines. Just 0 to 4 will do.
• The ACL for the management plane has an extra entry listing a single source address from which we will connect.

The setup part 2 – Hiding the backdoor

So far it’s still not special. Anyone checking the configuration can find this. But it can be altered using Embedded Event Manager.

These three EEM applets will filter out the commands and show a clean configuration instead!

• The “backDOORrun” is the main applet which replaces the standard “show run” by one that doesn’t list the rotary group, the extra ACL entry, the username and the EEM applets themselves. Note that it’s handy to name all objects part of the backdoor in a similar way, e.g. “backDOOR”, so they are matched with a single string.
• Since the above only affects “show run” two more applets are required for “show access-lists” and “show ip access-lists”. Note that these are only needed if a non-standard port is used, to mask the ACL.

Detectability

Several things do give away that there might be a back door. First of all, port 4362 will respond (SYN,ACK) to a port scan, revealing that something is listening. Second, although the commands are replaced, there’s a distinct ‘extra’ CLI prompt after the commands:

This only shows if you don’t do any pipe commands yourself and easily mistaken for an accidental extra hit on the ‘enter’ key, but when you’re aware of it, it does stand out.

And last, once you take the running config from the device (through TFTP for example) and open it in a text editor, everything will show as normal. And by knowing the EEM applet names, you can remove them.

Introduction to F5 network appliances.

I have to admit, this article will sound a bit like an advertisement. But given that Cisco has gotten enough attention on this blog already, it can only bring variation into the mix.

A short explanation of a series of different products offered by F5 Networks. Why? If you’re a returning reader to this blog and work in the network industry, chances are you’ll either have encountered one of these appliances already, or could use them (or another vendor’s equivalent of course).

LTM
The Local Traffic Manager’s main function is load balancing. This means it can divide incoming connections over multiple servers.
Why you would want this:
A typical web server will scale up to a few hundred or thousand connections, depending on the hardware and services it is running and presenting. But there may be more connections needed than one server can handle. Load balancing allows for scalability.
Some extra goodies that come with it:

• Load balancing method: of course you can choose how to divide the connections. Simply round-robin, weighted in favor of a better server that can handle more, always to the server with the least connections,…
• SSL Offloading: the LTM can provide the encryption for HTTPS websites and forward the connections in plain HTTP to the web servers, so they don’t have to consume CPU time for encryption.
• OneConnect: instead of simply forwarding each connection to the servers in the load balancing pool, the LTM can set up a TCP connection with each server and reuse it for every incoming connection, e.g. a new HTTP GET for each external connection over the same inbound connection. Just like SSL Offloading, it consumes fewer resources on the servers. (Not every website handles this well.)
• Port translation: not really NAT but you can configure the LTM for listening on port 80 HTTP or 443 HTTPS while the servers have their webpage running on different ports.
• Health checks: if one of the servers in the pool fails, the LTM can detect this and stop sending connections to the server. The service or website will stay up, it will just be able to accept fewer resources. You can even upgrade servers one by one without downtime for the website (but make sure to plan this properly).
• IPv6 to IPv4 translation: your web servers and entire network does not have to be IPv6 capable. Just the network up to the LTM has to be.

ASM
The Application Security Manager can be placed in front of servers (one server per external IP address) and functions as an IPS.
Why you would want this:
If you have a server reachable from the internet, it is vulnerable to attack. Simple as that. Even internal services can be attacked.
Some extra goodies that come with it:

• SSL Offloading: the ASM can provide the encryption for HTTPS websites just like the LTM. The benefit here is that you can check for attack vectors inside the encrypted session.
• Automated requests recognition: scanning tools can be recognized and prevented access to the website or service.
• Geolocation blocks: it’s possible to block out entire countries with automatic lists of IP ranges. This way you can offer the service only where you want it, or stop certain untrusted regions from connecting.

GTM
The Global Traffic Manager is a DNS forwarding service that can handle many requests at the same time with some custom features.
Why you would want this:
This one isn’t useful if the service you’re offering isn’t spread out over multiple data centers in geographically different regions. If it is, it will help redirect traffic to the nearest data center and provide some DDoS resistance too.
Some extra goodies that come with it:

• DNSSec: secured DNS support which prevents spoofing.
• Location-based DNS: by matching the DNS request with a list of geographical IP allocations, the DNS reply will contain an A record (or AAAA record) that points to the nearest data center.
• Caching: the GTM also caches DNS requests to respond faster.
• DDoS proof: automated DNS floods are detected and prevented.

APM
The Access Policy Manager is a device that provides SSLVPN services.
Why you would want this:
The APM will connect remote devices with encryption to the corporate network with a lot of security options.
Some extra goodies that come with it:

• SSLVPN: no technical knowledge required for the remote user and works over SSL (TCP 443) so there’s a low chance of firewalls blocking it.
• SSO: Single Sign On support. Log on to the VPN and credentials for other services (e.g. Remote Desktop) are automatically supplied.
• AAA: lots of different authentication options, local, Radius, third-party,…
• Application publishing: instead of opening a tunnel, the APM can publish applications after the login page (e.g. Remote Desktop, Citrix) that open directly.

So what benefit would you have from knowing this? More than you think: many times when a network or service is designed, no attention is given to these components. Yet they can help scale out a service without resorting to complex solutions.

Upgrading an IOS – The proper way.

Most readers of this blog will probably have upgraded an IOS on a Cisco switch or router already. Most of them will have done this using TFTP for the binary after erasing the existing IOS image, e.g.

Switch#delete flash:c3560-ipbasek9-mz.122-58.SE2.bin
Switch#copy tftp://192.168.1.1/c3560-ipbasek9-mz.150-1.SE.bin flash:c3560-ipbasek9-mz.150-1.SE.bin

And reload. The end. While the above works, it’s actually inefficiently and taking risks.

Types of releases
While taking the latest IOS right away may seem a good idea at first sight, there are some points to consider: are you looking for new features, or bug fixes? As with the above example, you would be making a jump from version 12.2(58) to 15.0(1). While this will introduce a lot of new features, it will also introduce a lot of new bugs.
Small releases only contain bug fixes. For example, upgrading from 15.0(1)SE to 15.0(1)SE3 will include many bug fixes. It’s therefore often better to wait out a new mayor release until some of these bugfix-releases are available. Be sure to read the release notes.

Checksum verification
Another thing that is usually forgotten is a checksum verification to see if the IOS isn’t corrupted. The IOS has a build-in command for this:

Switch#verify /md5 flash:c3560-ipservicesk9-mz.150-2.SE5.bin f73e32e66719fb48b11c849deee958e1

This will compare the md5 hash with a given one in the command. The original hash can be found on the Cisco website on the IOS download page, or by using a tool for this, for example WinMD5.

Proper boot flag
Uploading a new IOS doesn’t mean the router will use it automatically. It will follow the boot path:

Switch#show boot | include BOOT
BOOT path-list      : flash:/c3560-ipservicesk9-mz.150-2.SE5.bin

If it can’t find the file specified here, it will look for any .bin file on the flash and load the first one it finds. If you erase the old IOS before uploading a new one, the new IOS is the only one on flash and will be loaded. Still, it’s better to set the proper boot path:

Switch(config)#boot system flash:/c3560-ipservicesk9-mz.150-2.SE5.bin

Archive download
The copy command isn’t the only one to upload a new IOS. There’s a specific command to do an IOS upgrade:

Switch(config)#archive download-sw tftp://192.168.1.1/c3560-ipservicesk9-mz.150-2.SE5.tar

Note that it will require a .tar file, available from the Cisco website. The .tar file will automatically be unpacked. The command will also automatically change the boot path after a successful download and verify the IOS image by downloading the image two or three times and comparing the md5 checksums.

Internal copy
For a 3750 switch stack, the master switch will automatically push the newest IOS to the members of the stack as long as the IOS is in the same tree, e.g. it will automatically upgrade members from IOS 15.0(1)SE to 15.0(1)SE3 if the master has been upgraded to 15.0(1)SE3, but it will not do this between mayor versions like from 12.2(58) to 15.0(1).
However, once a file is present on one of the stack members, it can be copied to the other ones fast:

Switch(config)#copy flash3:c3750-ipbasek9-mz.150-2.SE5.bin flash2:c3750-ipbasek9-mz.150-2.SE5.bin

The above copies the IOS from the third stack member to the second stack member. It saves time and bandwidth.

Secure Copy Protocol
This one really comes in handy over WAN links and through some firewalls. TFTP uses UDP and has no concept of windowing so if latency increases, transfer speed of the new IOS will drop quickly.

A TCP-based transfer protocol will be able to use windowing. An obvious choice comes to mind: FTP. However, FTP will explicitly need to be opened on a firewall between the download server and the Cisco device and some firewalls can’t see the random high port used for transfer in the data negotiation. Switches may also have ACLs that can’t change according to the FTP data port being used.

So this is where Secure Copy Protocol (SCP) comes into play: it tunnels a file transfer through SSH, which is often already opened on firewalls, and doesn’t use a random high port. It also takes advantage of TCP windowing. Finetuning of TCP parameters prior to downloading the IOS is recommended:

Switch(config)#ip tcp window-size 65535
Switch(config)#ip tcp selective-ack

The default window size is about 8kB. Selective Acknowledgements make the transfer more resilient to packet loss by still acknowledging part of a packet stream that still reached the receiver. Also, since SCP works through SSH, you can use the following command to define a source interface:

Switch(config)#ip ssh source-interface Loopback0

And finally, the actual install command:

Switch#archive download-sw scp://scpuser@192.168.1.1/c3560-ipservicesk9-mz.150-2.SE5.bin

The user needs to be included in the command, the password will be asked after executing it. As a download server any SCP-capable server will do. A Debian Linux will support SCP out of the box, for example. By creating a specific user for SCP, e.g. ‘scpuser’, you can use the home folder (/home/scpuser/) as file share while still limiting access to other parts of the Linux system. At the same time, you can easily upload files to the download server using software such as WinSCP.

The most important limitation is that you need a fairly recent IOS already running before SCP works. For the 3560 and 3750 series you need version 15.0(2)SE1 or higher.

An update…

Well, I haven’t been active here for a while now. The problem is that, while I’m still learning a ton of things, it’s a lot of small things that I find hard to put in a blog post. For example: IS-IS basics, working different spanning-tree types in a network, fine tuning Wireshark, exploring EEM, and wondering what CCIEv5 will bring.

Yes, I’m still going for those numbers one day. Speculations for CCIEv5 are removal of some older topics such as frame relay and more focus of newer technologies. Since I’m working with many newer technologies I’m hoping it will work to my advantage when it’s finally announced.

Honestly, I can’t tell when this blog will be updated again. I’m hoping to find time and inspiration soon, but 1) this blog is a nice collection of information already even if inactive, and 2) learning technologies myself is still more important than putting them on a blog of course.

Greetings to all readers!

ASA: nice-to-know features.

I’ve already made an introduction to the ASA, but when working with them on a regular basis, it’s nice to know some features that come with the product to explain how it reacts and help troubleshooting. So for the interested reader with little ASA experience, below a few features that have proven handy to me.

Full NAT & socket state
Most consumer-grade routers with NAT keep a NAT state table that keeps state only with the source socket . A socket is an IP address and port paired together. For example, the following setup:

When connecting to the web server, remote socket 198.51.100.5:80, a local socket, for example 192.168.1.2:37004 is created. The router will then do a NAT translation to its outside IP address (a NAT/PAT with overloading or hide NAT) to socket 203.0.113.10:37004. This means that if return traffic arrives for destination 203.0.113.10 port 37004, it will be translated to 192.168.1.2 port 37004. However, without stateful firewalling, any packet will be translated back in again on port 37004, regardless of source. This is how some software like torrent programs do NAT hole punching. Also, no matter how big the pool of private IP addresses, the public IP address translations have a maximum of about 64,000 ports available (okay, 65,535 technically but there are probably some reserved and a source port below 1,024 is generally not recommended).

The ASA handles this differently: in combination with the stateful firewall a full state is made for each connection, both source and destination socket. This means the above translation is still done but no return traffic from another source is allowed. On top of that, if another inside host makes a connection towards a different web server, the ASA can reuse that port 37004 for a translation. Return traffic from that different web server will be translated to the other inside host because the ASA keeps a full state. Result: no 64,000 ports per public IP address the device has, but 64,000 per remote public IP address! This allows for even more oversubscription of a single public IP address, assuming not everyone is going to browse the exact same website.

Sequence randomization
A bit further into layer 4: TCP uses sequence numbers to keep track of the right order in a packet flow. The initial sequence number is supposed to be random, but this is not often the case in practice. In fact, one quick Wireshark from a connection to Google gives me this:

The problem is that guessing sequence numbers allows an attacker to intercept a TCP connection or guess an operating system based on the sequence number pattern. That’s where the second nice-to-know ASA feature comes into play: sequence randomization. By adding a random number to each sequence number (the same random number for each packet per flow) it becomes impossible to guess the initial sequence number of the next connection, as well as difficult to do any OS fingerprinting based on it.

Inspect policy-maps
For someone not familiar with the ASA, this is often a point of trouble. By default the ASA has no awareness above layer 4. This means any information not in the UDP or TCP header isn’t checked. Examples are HTTP headers, the FTP port used for transfer (which is in the payload) and ICMP Sequence numbers.

ASA requires configuration of policy-maps for this. This is why by default ping requests through the ASA don’t work: it cannot create a state for it. And for HTTP inspect, it checks for proper HTTP headers as well as the presence of a user-agent header. This means non-HTTP traffic cannot be sent through port 80, and incoming telnets on port 80 towards web servers aren’t accepted either, preventing some scans.

Capturing
Finally, one of the most useful functions. While many other platforms with a Unix-based OS allow some form of tcpdump, Cisco does not support it. However, you can do some form of capturing on an ASA, even with proper filtering.

First configure the ACL that will be used as a filter, otherwise you’ll capture all traffic for that interface.

ASA#configure terminal
ASA(config)#access-list ExampleCapture extended permit ip host 172.16.16.16 any
ASA(config)#exit

Next, find the correct interface name: the ‘nameif’ because the usual interface name will not do.

ASA#show run int vlan16
!
interface Vlan16
nameif Internal
security-level 50
ip address 172.16.16.1 255.255.255.0

Now you can start and show the capture.

ASA#capture TestCap interface Internal access-list ExampleCapture
ASA#show capture TestCap
76 packets captured

1: 16:45:13.991556 802.1Q vlan#16 P0 172.16.16.249.44044 > 203.0.113.10.22: S 3599242286:3599242286(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
2: 16:45:14.035474 802.1Q vlan#16 P0 172.16.16.249.44044 > 203.0.113.10.22: . ack 1303526390 win 17520
3: 16:45:14.037824 802.1Q vlan#16 P0 172.16.16.249.44044 > 203.0.113.10.22: P 3599242287:3599242338(51) ack 1303526390 win 17520
4: 16:45:14.067196 802.1Q vlan#16 P0 172.16.16.249.44044 > 203.0.113.10.22: . ack 1303526754 win 17156
5: 16:45:14.072887 802.1Q vlan#16 P0 172.16.16.249.44044 > 203.0.113.10.22: P 3599242338:3599242898(560) ack 1303526754 win 17156
6: …

Note that traffic is seen in only one direction here. To see return traffic, add the reverse flow to the capture ACL as well. Unfortunately, the capture must stay running while watching the output here. The capture can be stopped as following:

ASA#no capture TestCap

This will erase the capture also, so the show command will no longer work.

Additionally, you can do a real-time by adding the parameter ‘real-time’, but it’s a bit more tricky. This is not recommended for traffic-intensive flows, but ideal to see if a SYN is actually arriving or not.

ASA#capture TestCap interface External access-list ExampleCapture real-time
Warning: using this option with a slow console connection may
result in an excessive amount of non-displayed packets
due to performance limitations.

Use ctrl-c to terminate real-time capture

1: 16:45:51.755454 802.1Q vlan#16 P0 172.16.16.16.43969 > 203.0.113.10.22: . ack 2670019600 win 16220
2: 16:45:51.768698 802.1Q vlan#16 P0 172.16.16.16.43969 > 203.0.113.10.22: . ack 2670019768 win 17520
3: 16:45:51.768774 802.1Q vlan#16 P0 172.16.16.16.43969 > 203.0.113.10.22: . ack 2670019968 win 17320
4: 16:45:51.777501 802.1Q vlan#16 P0 172.16.16.16.43969 > 203.0.113.10.22: . ack 2670020104 win 17184
5: …

Just don’t forget to remove the ACL after you’re done.

CCDA certified!

Yes, another certificate! I took the exam last Monday and I passed. Although I have to admit, I didn’t find this exam easy. At all. But it’s have to compare with past exams because I haven’t taken one in the last year and a half.

That’s also the reason why: my previous certificates will expire in a little over a year. I still want that CCIE but I’m becoming uncertain if I will get it by then, and my company expects a certificate every now and then.

Up next: most likely CCDP. It will require just one exam, ARCH, since I passed ROUTE and SWITCH less than three years ago, and it will recertify my CCNP as well.

As for the CCDA content: interesting and I really learned things, but it’s not one I’d recommend for any engineer. The Routing & Switching track is much more of a challenge.

Best luck to all studying out there!