By Veghead
As a system administrator in a large academic institution, its obvious to me that network security should be a high priority when planning a network strategy. However, its not the system administrators that make the decisions and while most managers are willing to pay lip service to the need for security, they are far less willing to actually start writing cheques.
After a couple of unsuccessful crack attempts (and a couple of partially
successful ones) we were able to persuade them that we needed a firewall.
This in itself was an achievement. However even at the best of times, the
world of UK academia moves at a snail's pace and I for one wanted something
in place very quickly.
Our network consists of an extremely expensive ATM backbone running Lan-Emulation (LANE) out to switched 10M ports. The connection to the Internet consists of a single CLIP (Classical IP) PVC (Permanent Virtual Circuit) over a 34M ATM link from a metropolitan area network. Once inside our network, the PVC enters our central ATM switch, where it is converted to 155M LANE.
Our ATM vendors suggested that the best solution for us would be to use Firewall-1 on NT with two ATM adapters.
Personally, I dislike ATM as a way of carrying IP, and dislike LANE even more. There are plenty of reasons for this that run beyond the scope of this article but I was keen to avoid its use in a firewall and the thought of a firewall on NT turns the blood cold. So it was time to take action. We discussed various possible methods of creating an ethernet level "gap" between our external connection and the main part of the network, which would allow us to implement a nice bridging firewall with OpenBSD. However, after many discussions it was becoming apparent that our network equipment was incapable of letting us do this, and ATM was the only way around it.
Anxious to avoid becoming ensconced in a proprietary quagmire, I suggested buying an ATM adapter and testing OpenBSD's built-in, open, firewalling capabilities. Sadly the ATM support in OpenBSD was only able to work with two families of ATM adapter, both of which we were unable to obtain easily. The only cards we could easily obtain were the ones recommended by our vendors, the FORE 200e, which of course only provided drivers for SPARC Solaris and NT. Whilst I was aware that Linux also supported these cards, the idea of having core equipment running pre-alpha software did not inspire confidence.
It was looking like all was lost and we would yet again have to capitulate to the proprietary dictators when by accident I noticed a mention of the FORE ATM adapters on the FreeBSD site under a different ATM subsystem known as HARP. After some rapid research it became apparent that FreeBSD would support it perfectly well.
So, we ordered two adapters (one as spare) and I set about installing FreeBSD on a Pentium II 433 we had put aside for the purpose. Half an hour later, the box was up and running with the ATM adapter and a 3Com ethernet adapter installed. Ideally I'd have preferred to use an Intel EtherExpress Pro as past experience has made me lose confidence in 3-com adapters under high load.
The next step was to build a kernel that recognised them. Luckily, being BSD, there was plenty of useful documentation on the ATM subsystem contained in man pages and in /usr/share/examples/atm.
Using a copy of the GENERIC kernel config , I commented out pretty much everything except for essentials and added the following from LINT:options ATM_CORE #core ATM protocol family options ATM_IP #IP over ATM support options ATM_SIGPVC #SIGPVC signalling manager options ATM_SPANS #SPANS signalling manager options ATM_UNI #UNI signalling manager device hfa0 #FORE PCA-200E ATM PCIAlso, as this machine was eventually going to be firewalling I added support for ipfilter:
options IPFILTER
options IPFILTER_LOG
The choice of ipfilter over ipfirewall was purely personal. I'd used ipfilter before on OpenBSD and liked it.
Additionally, I upped MAXUSERS and the number of mbuf clusters so that high levels of traffic wouldn't be a problem:maxusers 128 options NMBCLUSTERS=4096The kernel was compiled, installed and booted. It came up cleanly, and obediently recognised the FORE adapter.
The FORE 200e adapters require microcode to be uploaded every time that it is powered up. The documentation in /usr/share/examples/atm provided detailed instructions on obtaining the binary code from the FORE web-site and installing it. The binary itself consists of a small file called pca200e.bin which is uploaded to the card using a command fore_dnld, a standard part of the FreeBSD distribution. In true, friendly, BSD style the rc scripts were already ATM aware and very little needed to be altered to configure the subsystem including uploading the microcode.
In order to describe the workings of the HARP ATM subsystem, here are the options in /etc/defaults/rc.conf for our setup. After each option is a description of what it doesatm_enable="YES"Causes the rc.atm to be executed with the atm settings contained in rc.conf. The pca200e.bin file, if present in /etc will be uploaded to all installed adapters automatically at boot if this is set to YES.
atm_netif_hfa0="fa 1"
Each Fore 200e has a "physical" interface name consisting of "hfa" followed by a number. The first interface is hfa0, the second hfa1 and so on. These interfaces are unusual as they don't correspond to the normal network interfaces such as xl0 or ppp0. Instead, each adapter must have one or more "logical" interfaces (confusingly referred to in the atm documentation as a "netif") which can be given their own IP addresses and be manipulated as normal interfaces with ifconfig and route etc.
The user decides the names given to these virtual interfaces. In the configuration line above, I have stated that I wish the interfaces to be called "fa", and only one to be initialised. This interface will therefore be called "fa0".
If we had wanted multiple logical interfaces for the ATM adapter, we could have done something like: atm_netif_hfa0="fa 5" which would have created 5 "netif"s called fa0, fa1,fa2,fa3 and fa4. Each of these could have then been given a different IP address with ifconfig.atm_sigmgr_hfa0="sigpvc" # Signalling manager for physical interface.
This line indicates that we will only be using PVCs with this interface. ATM has many modes of operation and each interface must be bound to a "signalling manager" that controls how it will be used.
The HARP system supports four different signal managers, all of which support PVCs.
| Signalling Manager | Description |
| SIGPVC | PVCs only |
| SPANS | FORE's proprietary signalling protocol |
| UNI30 | UNI 3.0 - the ATM forum's ATM User-Network-Interface Specification Version 3.0 |
| UNI31 | UNI 3.1 - the ATM forum's ATM User-Network-Interface Specification Version 3.1 |
atm_macaddr_hfa0="NO" # Override physical MAC address (or NO).Each 200e comes with its own MAC address, however this line allows you to override this with an alternative. We'll stick to the factory preset.
#atm_prefix_hfa0="ILMI" # NSAP prefix (UNI interfaces only) (or ILMI).
#atm_arpserver_atm0="0x47.0005.80.999999.9999.9999.9999.999999999999.00" # ATMARP server address (or local).
#atm_scsparp_atm0="NO" # Run SCSP/ATMARP on network interface (or NO).These are commented out as we are not using UNI and thus don't need an ATM address. The scope of this article doesn't allow a full description of these.
atm_pvcs="lman"Here we have created a whitespace separated list of PVCs that we define further down. In our case we only have one.
atm_arps=""Again, we don't need to use ATMARP as we are using PVCs.
atm_pvc_lman="hfa0 0 126 aal5 snap ip fa0 AAA.BBB.CCC.DDD"Here is where we define our pvc which is arbitrarily called 'lman'. The content consists of arguments to the command "atm pvc". In a nutshell, this line may be read as:
With this being such a core piece of equipment, we wanted to satisfy ourselves that it could stand up to the job. We also wanted to ensure that the ATM side of things was going to work nicely with our network and with ipfilter. So as an initial test, we constructed an identical FreeBSD box with the spare ATM adapter. Both boxes were then connected to each other via the ATM switch, and a PVC between them established.
It wasn't so much that we didn't expect it work, more that we didn't
expect it to work first-time. But it did. Perfectly. Several nasty traffic
generation scripts were then constructed and executed on both machines
to really hammer the link, and everything continued to work perfectly.
So we then started to play around with ipfilter to ensure it co-operated
with the ATM system properly. Again, it worked perfectly no matter how
harshly we treated it.
Now that we had satisfied ourselves that it was all going to work properly the next stage was to decide on a method of implementation. It was decided that we would initially run it in place with no firewalling for a couple of days, to ensure that it was happy with the bandwidth. After this we could slowly introduce rules.
Due to our network configuration, the firewall would effectively become
the border router, routing between the CLIP PVC and our LANE backbone.
Beforehand, routing was achieved by one of the FORE Powerhubs which formed
part of the LANE network.
The ethernet adapter plugged directly into one of the switched LANE
ports, and the ATM adapter connected to the incoming PVC via the main ATM
switch (this was necessary as the incoming connection was 34Mb and the
switch allowed use to convert it to standard 155Mb).
We closely monitored all traffic for a couple of days until we were sure there was no detrimental effect on bandwidth after the implementation of the firewall. It could have been my imagination, but it actually seemed to improve performance. Regardless, there was certainly no noticeable reduction in performance.
The next stage was to implement a set of light rules that would block some really obvious holes such as IP spoofing and smurfing. We based our initial ruleset around the example sets provided as part of the OpenBSD distribution in /usr/share/ipf (available from the OpenBSD web site). Once we were happy with our rules, we introduced them and monitored the filter with ipmon(8). In fact we did notice a few legitimate packets being blocked and traced the problem to a small typo in the rules file. It was a simple matter to fix and of course no interruption in service was required to do so.
We have been running the firewall in place for several months now (without a reboot - its not NT), with the "light" firewalling rules in place and have had no problems. The throughput is equally as good as before and we frequently have periods of extremely high load which FreeBSD seems to cope with admirably. We have also observered (with ipmon) many legitimate blocks of scans and unauthorised NFS mount attempts.
We have delayed instituting firmer rules until we are sure we have accounted
for all internal machines that require direct access to the outside world.
Indeed, the political aspects of this project proved more of an obstacle
than either technical or financial constraints.
The process of auditing all machines in the institution has highlighted
just how many ad-hoc servers have been established, independently of the
computer services department, on which people have begun to rely. The Firewall
is also providing us with a perfect opportunity to audit and rationalise
them.
Overall FreeBSD has provided us with a quick, powerful and secure solution. Past experiences with other commercial firewalls has certainly shown me that security is certainly not proportional to price. Knowing exactly what's going on in the FreeBSD box gives me far greater confidence in the security of the firewall than I've had with any other proprietary product.
Back