IPIP tunnel to SLiRP

I know this is what 99.99% of people hope I never do, but let’s make an incredibly insecure VPN! yay!

Motivation

So the thing is that I have a cisco router and I’d love for it to connect to some Windows machine over an existing OpenVPN, and NAT out the Windows side.  Except for getting the VPN installed, they won’t give me anything else.  And they SURE as heck won’t let me connect a cisco router up…..

So first things first, I need to configure my cisco router for an IPIP tunnel, to my test Windows machine, and use the SLiRP default addresses:

interface Tunnel0
description “SLiRP tunnel”
ip address 10.0.2.15 255.255.255.0
ip mtu 1452
tunnel source GigabitEthernet0/1
tunnel destination 192.168.1.10
tunnel mode ipip
end

Now to start programming.

Well then I went looking and found this fun filled page, about calling winioctl’s myself, and getting winsock to do all kinds of fun things.  Namely how IPIP actually works, as it’s is it’s own protocol (none of that pesky TCP/UDP it’s IPIP!) and more importantly I can receive the traffic.

So looking at a quick UDP client/server I figured out that I can modify that so instead of listening with UDP like this:

if((s = socket(AF_INET , SOCK_DGRAM , 0 )) == INVALID_SOCKET)

I can instead call for a RAW socket, and listen on protocol #4 aka IPIP.

if((s = socket(AF_INET , SOCK_RAW , 4 )) == INVALID_SOCKET)

One caveat I had on this, is that you need to run as Administrator on the Windows machine to create raw sockets.  If you don’t have administrator privleges you’ll get this error:

Could not create socket : 10013socket() failed with error code : 10013

Now add in some nonblocking, and feed the data into SLiRP, and I got invalid data!  Using wireshark I can see that I only receive the IP portion of the data, so no hardware frame, but what is more interesting is that I receive ALL of the IP information so I get the IP+IP+DATA.  So I have to forge a L2 header, and cut out the first IP header.  I did this by cheating, using the following for a L2 header:

0x44,0x8a,0x5b,0x62,0xcb,0xaf,0x00,0x1b,0x90,0x21,0x58,0x1b,0x08,0x00

I then just memcpy that to the start of my buffer, then copy in the rest of the received data like this:

memcpy(buf2+14,buf+20,recv_len-20);

And now I can forge data going to SLiRP to make it happy!

And sending replies didn’t make wireshark happy at all, as there is an L2 header in there, that just doesn’t make sense in L3 space, so I trimmed that with the following:

memcpy(buf,qp->data+14,qp->len-14);

Putting it all together

And now much to my amazement I can ping SLiRP from my 7206!

IPIP Ping!

Ok, I know what you are thinking. ICMP is great, but how about TCP?  Can I actually use this thing?

I add a route to my BBS over the SLiRP tunnel, with a simple route statement:

ip route 172.86.181.35 255.255.255.255 10.0.2.2

and then telnet…

Telnet to my BBS over IPIP to SLiRP

So yes, it does actually work!\

I don’t think anyone will ever want to use this, but for me it’s 100% novelty in that I could.

Executable & source code is here, ipip.7z.

I suppose later I could look at ipdecap, to work out how to work with GRE.

The client needs to access the internet!

But let’s not give them access to everything.

This is a common scenario I see, where someone needs to get updates to some magical software package on the internet.  Great.  And people just give them access to ANY site, which ends up being not only the internet (the intended destination) but the rest of their internal network.  Granted a good defense in the SDN world is inbound rules as well for each VM, but everything is never 100%.

RFC1918 defines our friends, the private address ranges:

     10.0.0.0        -   10.255.255.255  (10/8 prefix)
     172.16.0.0      -   172.31.255.255  (172.16/12 prefix)
     192.168.0.0     -   192.168.255.255 (192.168/16 prefix)

However the solution to this fun filled problem is to grant them HTTP/HTTPS access to the inverse of this.  Enter the netmask command.  You can give it a range, and it’ll lay out what networks to you need to add like this:
     netmask -c 0.0.0.0:9.255.255.255
     0.0.0.0/5
     8.0.0.0/7

Now I can exclude everything right up until 10.0.0.0/8 !

It’s quite the handy tool, but I didn’t see any Windows version.  So a few minutes with MinGW, and dealing with it’s weird Makefile’s way of linking things, and here you go!

This way you can permit internet access, not give them inside, access and still have a global DENY actually work.

YAY.

And if anyone is interested here are the networks:

     0.0.0.0/5
     8.0.0.0/7
    11.0.0.0/8
    12.0.0.0/6
    16.0.0.0/4
    32.0.0.0/3
    64.0.0.0/2
   128.0.0.0/3
   160.0.0.0/5
   168.0.0.0/6
  172.0.0.0/12 
 172.32.0.0/11
 172.64.0.0/10
 172.128.0.0/9
   173.0.0.0/8
   174.0.0.0/7
   176.0.0.0/4
   192.0.0.0/9
192.128.0.0/11
192.160.0.0/13
192.169.0.0/16
192.170.0.0/15
192.172.0.0/14
192.176.0.0/12
192.192.0.0/10
   193.0.0.0/8
   194.0.0.0/7
   196.0.0.0/6
   200.0.0.0/5
   208.0.0.0/4

Yes, I know it’s a LOT of typing.

Dynamips on MinGW

It’s always bugged me that the only way to build Dynamips for Windows was with Cygwin.

Well fear no more, I’ve mashed an old version (I would have tried newer, but of course Cmake fails spectacularly and with zero help as always!) and not only does it compile, but it can boot a 7200 version of IOS.

Dynamips on MinGW

Dynamips on MinGW

JIT is broken.  You have to telnet into the console.  And the console is a little wonkey as I’m sure it’s doing a lot more UNIX translation vs being a Win32 program but it does work enough to login, save the config, and reload.

But it’ll crash on reload.

I’m sure it’s full of bugs actually.

https://sourceforge.net/projects/dynamips-mingw/

I started with Dynamips 0.2.8-RC7-community and started commenting out stuff to get it to compile.  Luckily I found this ezwinports that includes mairix that includes some memory mapping functions, namely mmap and munmap ported to Win32 in an early glibc port. While I was trying to integrate libuuid, I got this fun error:

mingw “error: conflicting types” “UUID”

MinGW includes UUID support, since it’s a Microsoft thing.  Unfortunately libuuid doesn’t include unique names,  so I had to rename uuid_t to uu_uuid_t

//typedef unsigned char uuid_t[16];
typedef unsigned char uu_uuid_t[16];

in the uuid.h header file, along with all instances in Dynamips.

I also borrowed sendmsg/recvmsg along with the msg structures from VLC.  inetaton.c from WSHelper, and finally telnet.h from NetBSD.

After that it was a matter of making sure Winsock starts up, and fixing some linking breakage.

For those who want to try, the binary package is here.  I’ll have to setup git on this machine and upload all the changes.  It shouldn’t require any DLL’s, although I haven’t looked at the pcap stuff, as I mentioned it’s largely untested, so I have no idea if any of it works other than the telnet console.

Random links

No I’m not dead, just been busy.

But here is some interesting things I’ve seen the last while:

Infer: static code analysis from facebook of all people.  Supports C, Objective-C and Java.

Dr Jack Whitham’s blog, with some interesting stuff related to compiler optimizations and how they alter floating point results, along with ‘bug 323‘, and some DOOM fun!  Plus he has his updated source repositories online here.

And finally, Building A 10BASE5 “Thick ethernet” network.  A fun look at the first gen ethernet cabling on ‘slightly’ newer machines.

GNS3 + VMWare +Netware 3.12

So continuing from where I left off, I thought I’d setup a small Netware network onto my GNS3 testbed.

Netware was once the big dog in the networking world, but now it’s just a footnote in obscurity.  Long gone are the days of syscon, rconsole, and all the frametypes that Netwaer supported in equally incompatible ways.  So maybe it’s a good thing.

But I digress, years ago I paid good money for a 50 user version of Netware 3.12.  I don’t know why either, but I occasionally like to get it running so I don’t feel like I completely wasted the thing.

With the right lan drivers, installation is a snap.  Or if you don’t have the right lan drivers, create a MS-DOS VM, so it’ll have an IDE hard disk.  You can install netware without lan drivers just fine.  I just installed netware by itself, then added in some patches for the network card & the idle program so it wouldn’t eat 100% of my CPU.  For the network though the command line was a little weird but I worked it out

file server name vmware
ipx internal net 538787d
mount all
load c:\server.312\nw4-idle.nlm
load c:\server.312\pcntnw slot=10001 FRAME=Ethernet_802.2
BIND IPX to pcntnw NET=cab2
load remote password
load rspx

Which gets me networking, and rconsole support.  With the server running, I just then had to enable IPX routing on my cisco routers.
R1

ipx routing ca00.1b00.0008
interface FastEthernet0/1
ipx encapsulation SAP
ipx network CAB1
interface Serial1/0
mtu 2000
ipx network CAB0

R2

ipx routing ca08.140c.0008
interface FastEthernet0/1
ipx encapsulation SAP
ipx network CAB2
interface Serial1/0
mtu 2000
ipx network CAB0

 Now with this all in place I can see the IPX routes from R1, including the one that goes directly into the server:

R1#show ipx route
Codes: C – Connected primary network, c – Connected secondary network
S – Static, F – Floating static, L – Local (internal), W – IPXWAN
R – RIP, E – EIGRP, X – External, A – Aggregate
s – seconds, u – uses, U – Per-user static/Unknown, H – Hold-down

4 Total IPX routes. Up to 1 parallel paths and 16 hops allowed.

No default route known.

C CAB0 (HDLC), Se1/0
C CAB1 (SAP), Fa0/1
R CAB2 [07/01] via CAB0.ca08.140c.0008, 56s, Se1/0
R 5383787D [08/02] via CAB0.ca08.140c.0008, 56s, Se1/0

And of course I can see the server!

R1#show ipx servers
Codes: S – Static, P – Periodic, E – EIGRP, H – Holddown, + = detail
U – Per-user static
2 Total IPX Servers

Table ordering is based on routing and server info

Type Name Net Address Port Route Hops Itf
P 4 VMWARE 5383787D.0000.0000.0001:0451 8/02 2 Se1/0
P 107 VMWARE 5383787D.0000.0000.0001:8104 8/02 3 Se1/0

Now if you remember from the bad old days of Netware, each service running on a server is advertised.  So in this case, type 4 is a file/print server, and 107 is a server running the ‘remote’ facility so you can rconsole into it.

Another thing to take note, is that because I configured IPX on serial interfaces is that their encapsulation is always HDLC.  Also because IPX cannot fragment like TCP/IP the MTU on the serial interface *MUST* be larger than ethernet’s 1500.  I like 2000 as I avoid all frame overhead this way.

Now I’m using the old netware client with the drivers I found here, and it works fine, as long as I had the MTU bumped up on the serial interface.  Now I could have used ethernet, and it’d achieve the same thing, except I’m not in some weird MTU war.

IPX network

IPX network

SA-COMP/1

SA-COMP/1

At the same time, it is very slow.  If you want to do anything reasonably fast, you really need to move your client to the same VNet as the server.  But if you want to feel like the old days when connections were slow, well you’ll enjoy a max 400k connection.  I tried enabling stacker compression on the interface, but it resulted in packet corruption.  I think it’s an IOS thing though, so if you play with this, you may have better luck.  Back before MPLS was everyone’s latest wan joy, I ran stacker compression, and the speed boost was really noticeable, with a 1.5:1 or 2:1 compression not being uncommon.  Although on lower routers it could be a killer (the 2500…) or a 7000 with multiple interfaces doing compression it could also bring a router to it’s knees.  Also looking at the support table for the  SA-COMP/1  SA-COMP/4 I doubt 12.4 would support stacker compression properly.

Platform Recommended Minimum Cisco IOS Release
Cisco 7000 series and Cisco 7500 series With VIP2-40(=) Cisco IOS Release 11.1(6)CA or a later release of Cisco IOS Release 11.1 CA
Cisco 7000 series and Cisco 7500 series With VIP2-50(=) Cisco IOS Release 11.1(14)CA or a later release of Cisco IOS Release 11.1 CA
Cisco 7204 and Cisco 7206 Cisco IOS Release 11.1(6)CA or a later release of Cisco IOS Release 11.1 CA1
Cisco 7204 and Cisco 7206 Cisco IOS Release 12.2(4)B or a later release of Cisco IOS Release 12.2 B

Otherwise, yeah it works fine.  I also loaded up a Windows NT client, and it works too!

GNS3 & VMware

Well I’ve been back using GNS3 to simulate some networks professionally.  And well, I hit a roadblock of a strange kind.

In my “LAB” I want to have a ESXi host talking to vCenter, and I wanted to setup a custom logging program which logs to MSSQL, and Maybe Oracle.  For routing I need Junos and cisco IOS.

Now the problem is that takes a little bit of everything.  The Qemu bundled with GNS3 horribly ancient, and my attempt to drop in Qemu 1.6.0 just fails.  Also running things like Windows Server 2008r2 and ESXi run best under VMware.  The SQL server stuff can be any version, so even NT 4.0/SQL 7 is fine which GNS3’s Qemu can run, but it is kind of slow.  Which I know it isn’t fair to compare something like Virtual Box to VMWare.

So ideally the best  bet is to tie them all together, and I found a way.

First I’m using VMWare Player version 6.0.1 build-1379776.  The people financing this insane project have things like VMWare workstation but I have to download it through them, and their link is insanely slow, so I’m sticking with the player for now.  But I was able to persuade a user to extract “VMware Workstation 10.0.2 build 1744117”, and retrieve two files for me,  vmnetcfg.exe and vmnetcfglib.dll.  So this way I can setup VMWare network interfaces which GNS3 can then latch onto with pcap.

The feature was available in earlier versions of VMWare Player, but the needed files were removed in the latest version.  However I found if you have access to a new version of VMware Workstation you can just snag these two files, and run them like this (as administrator):

rundll32.exe vmnetcfglib.dll VMNetUI_ShowStandalone

And now you can setup some networks.

x

VMware’s default network setup

Now from what I have read online, the best thing to do is leave the default networks alone.  However I found that if I left VMNet0 to the defaults I was unable to join any VM’s to the different networks.  So I bind this to my physical Ethernet connection.

Now I want to have various network points to attach VMs onto.  The best part that I’ve found is that pcap works on these networks for both listening and injection, so these make better ‘hub’ inspection points IMHO.  Also this means you can run emulators that inject libpcap as a method of communication (SIMH, WinUAE, and even my ancient Qemu 0.9.0…)

The big thing is when adding networks, DISABLE DHCP.  You can leave the rest where it is, it really doesn’t matter.

With Networks

With Networks

As you can see, I’ve now added VMnet2 – VMnet7.  This should give me enough user networks for now to play with.  I’ve also unchecked the local DHCP service, as I may want to run my own DHCP on an emulated server to make sure DHCP relay works through my virtual network.  Once you are happy, you can hit Apply and it will create the network interfaces on your computer.

So many networks!

So many networks!

Now going into the control panel, and looking at the network adapters (search for “view network connections”), and you will see there are a bunch of these VMWare Network Adapters.  The worst part is that they all have full networking enabled, which we don’t want. So starting with VMnet2, we need to unbind all high level access.

Before

Before

I unbind the ‘Client for Microsoft Networks’, ‘File and Printer Sharing for Microsoft Networks’, and TCP/IP version 4 and version 6. This the Link-Layer topology discovery stuff.  I also enabled the VMware Bridge Protocol.

After

After

Now I just have to repeat this for each of the adapters that we installed, in this case VMNet2 – VMNet7.  Remember to leave VMNet0, VMNet1 and VMNet8 alone!

Now for the real fun, you have to reboot for the changes to take effect.

After a reboot, if you run Wireshark, you should now see all the interfaces!

Wireshark  with the new interfaces

Wireshark with the new interfaces

Ok so far, so good, but let’s tie this mess together!

HHGTTG refrence

HHGTTG refrence

So let’s build a network.  Our “home” site will have a server network with the ESXi server serving some virtual servers, a user network which will contain our management workstation & a MS-DOS netware machine.  We will then have a remote network with different machine types, which will be a 4.3BSD VAX, and an Amiga running NetBSD.  We’ll also include a Novell Netware 3.12 server.  Add in an ‘internet’ router, and we should be good.

Clouds

Clouds

The first step is to create some clouds.  Each one of these will then be associated with a VMNetwork device.

Network Device
Server VMNet2
Vax-Amiga VMNet3
NetwareServer VMNet4
Client VMNet5
Internet VMNet8

Now you’ll notice that I’m assigning VMNet8 to the ‘internet’.  If you remember the original VMWare table, the VMNet8 device is a ‘NAT’ device.  So we can use that to get to the internet (well anything else the base device can access).  Now I’ve gone ahead and added in two cisco routers, a single juniper router and a hub, as the juniper device cannot directly connect to the Internet cloud, but using the hub for the intermediary is ok.

Add in the routers & hub

Add in the routers & hub

Now it’s time to add some interfaces to the routers.  I’m going to put the C7200-IO-2FE into both R1 & R2, along with a PA-4T+ serial adapter.  Because I want to pretend to have a fast internet connection I’m also going to place a PA-GE interface into R1 in slot 2.

Now we need to bind each cloud to the corresponding VMnet interface.

Server Network

Server Network

Simply double click on the cloud, and select the VMNet interface from the drop-down list, then add it and the interface is now bound.  Repeat for each of the clouds.

Now we can connect the interfaces.

 

Now with Interfaces

Now with Interfaces

So R1’s FA0/0 is connected to the SERVER cloud, FA0/1 is connected to the CLIENT cloud. Serial 1/0 is connected to R2’s Serial 1/0 interface. the G2/0 interface is connected to Junos1’s e1 interface

R2’s FA0/0 is connected to the VAX-AMIGA cloud, and the FA0/1 is connected to the NETWARESERVER cloud.

Junos1’s e0 is connected to HUB1, which is then connected to the INTERNET cloud.

NAME Interface IPX IP
Server VMNet2 192.168.0.0/24
Vax-Amiga VMNet3 192.168.1.0/24
NetwareServer VMNet4 cab2
Client VMNet5 cab3 192.168.2.0/24
Internet VMNet8 DHCP
serial none cab0 192.168.255.0/30
GigE none 192.168.255.4/30
Configure the network interface in VMWare Player

Configure the network interface in VMWare Player

Since I already have an ESXi VM on Player, I’m going to use this for my illustration.  All I need to do here is change the existing network from being ‘bridged’ on my native network, to now being on VMnet2, which now places it inside of my GNS3 world.  Likewise I take a Windows XP client, and place it on VMnet3.

Now to configure R1 like the following:

Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 192.168.0.1 YES manual up up
FastEthernet0/1 192.168.2.1 YES manual up up
Serial1/0 192.168.255.1 YES manual up up
GigabitEthernet2/0 192.168.255.5 YES manual up up

And R2 like the following:

Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 192.168.1.1 YES manual up up
FastEthernet0/1 unassigned YES unset up up
Serial1/0 192.168.255.2 YES manual up up

And now I can connect from my Client PC, the VMware ESXi server!

Connection

Connection

This gives me an easy way to ‘view’ into what is going on for my client to connect to the server.

Now some quick EIGRP to get R1 & R2 routing together..

R1:

router eigrp 1
network 192.168.0.0
network 192.168.2.0
network 192.168.255.0 0.0.0.3
no auto-summary

R2:

router eigrp 1
network 192.168.1.0
network 192.168.255.0 0.0.0.3
no auto-summary

And now we can check routs on R2, and see it’s learnt the routes from R1:

  192.168.255.0/30 is subnetted, 1 subnets
C 192.168.255.0 is directly connected, Serial1/0
D 192.168.0.0/24 [90/2172416] via 192.168.255.1, 00:15:50, Serial1/0
C 192.168.1.0/24 is directly connected, FastEthernet0/0
D 192.168.2.0/24 [90/2172416] via 192.168.255.1, 00:15:50, Serial1/0

I was a little disappointed though, that Olive can’t do any flow based stuff like security policies or NAT.

So onward with SIMH.  I’ve found that I have a LOT of Ethernet interfaces and some things cannot deal with that.  I had to make a trivial change to sim_ether.h:

#define ETH_MAX_DEVICE        32                        /* maximum ethernet devices */

SIMH had this value hard-coded to 10, and it crashed because I have… 11 interfaces.  So it just took a quick re-compile and now I can see my interfaces!

0 \Device\NPF_{0C6D7EF7-30D4-4AB0-AB3E-AC6EAB42B9C5} (VMware Network AdapterVMnet2)
1 \Device\NPF_{1A17F8DF-DC65-420E-9A7A-3F8D22EC0D12} (VMware Network AdapterVMnet6)
2 \Device\NPF_{5A889C62-8180-4DB5-8FFE-3B6B8B9DFFAF} (VMware Network AdapterVMnet7)
3 \Device\NPF_{A6B89C5C-C28C-424E-B795-F90F97FA0FE7} (VMware Network AdapterVMnet8)
4 \Device\NPF_{21FFD0D4-1B8B-47B7-B0DD-28CD67DF4080} (Local Area Connection)
5 \Device\NPF_{70AA2D26-7B96-42FB-9FA6-8A7386753099} (Local Area Connection 2)
6 \Device\NPF_{98F44EE6-626B-48CB-952D-9C890F44A4A5} (VMware Network AdapterVMnet5)
7 \Device\NPF_{D294A70E-07B3-4CA8-A88D-D6C392696E99} (VMware Network AdapterVMnet1)
8 \Device\NPF_{F746872D-7687-4867-958C-96A62BA5E284} (VMware Network AdapterVMnet3)
9 \Device\NPF_{783262C6-8B95-4F9E-B198-78E2D9B256BB} (Bluetooth Network Connection)
10 \Device\NPF_{D6726593-C290-4821-8D43-D180CF5631BA} (VMware Network AdapterVMnet4)

Wow!

So with SIMH I can now attach to eth8, which maps to VMnet3.

set xu ena
att xu eth8

 

Easy, right?

And even better, it works!

R2#ping 192.168.1.15

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.15, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/15/28 ms
R2#

And the XP workstation can telnet to it…

VMWare routed through GNS3

VMWare routed through GNS3

Next I’ll have to add in some NetWare fun.  For the heck of it.  Good news is that it works!

One caveat I’ve found is that sometimes the ARP response time isn’t so hot, and it seems like everything times out..  So you may want to tweek the default arp age on the cisco side (interface bla/arp timeout 600..?).

Using expect with Cisco IOS

Following up my JunOS post, here is a handy script I cooked up to pull the configuration from a Cisco IOS device.  The one trip up for this stuff is sometimes you can logon to a cisco device, and you can be at the enabled state, you may have to enable,  and depending on how it’s configured you may have to use an enable password, which may be your password (again) or you may have to use a different password.

So yeah with a bunch of testing around this seems to work well enough for me.

#!/usr/local/bin/expect —
set MYUSER “my_user_name”
set MYPASS “my_password”
set ENPASS “my_enable_password”

set HOST [lindex $argv 0];
set timeout 90
if {$argc!=1} {
puts “Usage is scritpname <ip address>\r”
exit 1
}

#
#
puts “Connecting to $HOST\r”

spawn ssh $HOST -l $MYUSER

# Deal with hosts we’ve never talked to before
# or just logon
#
expect {
“*yes/no*” {send “yes\r” ; exp_continue }
“*assword:” {send “${MYPASS}\r” }
}
set ALREADY 0
expect {
“\r*>” {}
“\r*#” { set ALREADY 1}
“*enied” {exit 1}
“*assword” {exit 1}
}

if { $ALREADY < 1 } {

send “enable\r”
expect “*assword:” {
send “${MYPASS}\r”
expect {
“*enied” {
send “enable\r”
expect “*assword:”
send “${ENPASS}\r”
expect {
“*enied” {
exit 1}
“\r*#” {}
}
}
“\r*#” {}
}
}
}

send “show run\r”

expect {
“ore” {send ” “; exp_continue}
“\r*#” {}
}

#Let’s get out of here
send “q\r”
expect eof
exit 0

 

This is a little more cleaner than the prior JunOS one, as I’ll keep on improving it.

It works with ASA’s (tested 8.2)and IOS (tested 12.2)

Doom IPX revisited

So, since I’ve been starting up my virtual network thing, I got distracted with this thread on reddit

Fun times ahead!

Fun times ahead!

, talking about the good old days of Doom.  Now Doom was revolutionary for the music, the sound effects, the 2.5D environment, the fast pace, the violence, the unlimited lives, dynamic lighting and of course LAN play.  Now what was cool is that it could easily run over IPX/SPX networks.  Although you did have to be on the same network it was very easy to setup, and get going. But as anyone can tell you, the moment you setup a four player deathmatch on a company LAN people would go nuts, as you would without a doubt bring the entire network to a halt.  Back when I used to play this we would just disconnect the 10base2 cable from the lab 486DX2/50 PC’s and just make our own four computer segment.  We did notice that if we had 3 four player games on the same network (afterhours!) that it was unplayable.  I never did have access to a sniffer back then, and when I got a corporate job, being caught with something like Doom was grounds for termination so I never did get to see what the big deal was.

So I figured I’d cook up a super quick lab to check it out.  This is my Dynagen configuration.  Sadly I do need a cisco router to preform the network capture, although I don’t need to configure it to see what is going on.

autostart = True

[localhost]

[[7200]]
image = ../C7200-JS.BIN
npe = npe-400
ram = 160
idlepc = 0x60529c84
disk0 = 0
mmap = False
ghostios = True

[[ROUTER corertr1]]
model = 7200
slot1 = PA-8E
F0/0 = coresw1 5

[[ethsw coresw1]]
1 = access 1 NIO_udp:41300:127.0.0.1:51300
2 = access 1 NIO_udp:41301:127.0.0.1:51301
3 = access 1 NIO_udp:41302:127.0.0.1:51302
4 = access 1 NIO_udp:41303:127.0.0.1:51303
5 = access 1

As you can see this is very simple, a switch with four ports setup for four virtual computers, and a fifth port for the router.  Next I run four instances of Qemu with a copy of Doom v1.1 shareware, and a IPX/SPX client.  I fire them up like this:

./qemu/qemu-system-i386 -cpu pentium -L ./qemu/pc-bios/ -m 16 -hda doom11-00.img -soundhw sb16,adlib -net nic,model=pcnet,macaddr=00:11:22:33:44:00 -net socket,udp=localhost:41300,localaddr=0.0.0.0:51300 &
#
./qemu/qemu-system-i386 -cpu pentium -L ./qemu/pc-bios/ -m 16 -hda doom11-01.img -soundhw sb16,adlib -net nic,model=pcnet,macaddr=00:11:22:33:44:01 -net socket,udp=localhost:41301,localaddr=0.0.0.0:51301 &
#
./qemu/qemu-system-i386 -cpu pentium -L ./qemu/pc-bios/ -m 16 -hda doom11-02.img -soundhw sb16,adlib -net nic,model=pcnet,macaddr=00:11:22:33:44:02 -net socket,udp=localhost:41302,localaddr=0.0.0.0:51302 &
#
./qemu/qemu-system-i386 -cpu pentium -L ./qemu/pc-bios/ -m 16 -hda doom11-03.img -soundhw sb16,adlib -net nic,model=pcnet,macaddr=00:11:22:33:44:03 -net socket,udp=localhost:41303,localaddr=0.0.0.0:51303 &

Again all very simple.

From there I configure each of the four VM’s for a four player game, start the game, then connect.

Loneliest deathmatch

Loneliest deathmatch

I only had the game running for a few seconds when I went to my Dynagen console, and setup a capture session:

=> capture corertr1 fa0/0 doom.cap
=>

As you can see very simple.  I walk the guys around a little bit, shoot someone, then just quit all the clients, and turn off the VM’s.  Then I end the capture

=> no capture corertr1 fa0/0
=>

And now I’m left with a 50MB file!

$ ls -alh *cap
-rw-r–r– 1 jsteve staff 50M 25 Oct 14:27 doom.cap

I’m shocked!  I wasn’t even running Doom that long!  A quick peek reveals why Doom was never popular on corporate networks.

14:25:57.464031 IPX 00000000.00:11:22:33:44:00.869c > 00000000.ff:ff:ff:ff:ff:ff.869c: ipx-#869c 444

14:27:28.996489 IPX 00000000.00:11:22:33:44:01.869c > 00000000.ff:ff:ff:ff:ff:ff.869c: ipx-#869c 444
14:27:28.997618 IPX 00000000.00:11:22:33:44:03.869c > 00000000.ff:ff:ff:ff:ff:ff.869c: ipx-#869c 444
14:27:29.019568 IPX 00000000.00:11:22:33:44:01.869c > 00000000.ff:ff:ff:ff:ff:ff.869c: ipx-#869c 444
14:27:29.026000 IPX 00000000.00:11:22:33:44:03.869c > 00000000.ff:ff:ff:ff:ff:ff.869c: ipx-#869c 444
14:27:29.026205 IPX 00000000.00:11:22:33:44:01.869c > 00000000.ff:ff:ff:ff:ff:ff.869c: ipx-#869c 444
14:27:29.054425 IPX 00000000.00:11:22:33:44:01.869c > 00000000.ff:ff:ff:ff:ff:ff.869c: ipx-#869c 444

If you see the destination it is ff:ff:ff:ff:ff:ff, a broadcast packet.  And I have 50MB of this in the span of a minute and a half!  Even worse, opening the packet stream most of it is empty packets!

14:26:22.806014 IPX 00000000.00:11:22:33:44:00.869c > 00000000.ff:ff:ff:ff:ff:ff.869c: ipx-#869c 444
0x0000: e0e0 03ff ff01 da00 0000 0000 00ff ffff …………….
0x0010: ffff ff86 9c00 0000 0000 1122 3344 0086 ………..”3D..
0x0020: 9c00 0000 0000 0000 0001 0000 0001 0000 …………….
0x0030: 0002 0000 0000 0000 0000 0065 0000 0000 ………..e….
0x0040: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0050: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0060: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0070: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0080: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0090: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x00a0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x00b0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x00c0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x00d0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x00e0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x00f0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0100: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0110: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0120: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0130: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0140: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0150: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0160: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0170: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0180: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x0190: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x01a0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x01b0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x01c0: 0000 0000 0000 0000 0000 0000 0000 0000 …………….
0x01d0: 0000 0000 0000 0000 0069 0000 00 ………i…

Needless to say this could have been optimized a little better.. but at the same time back in 1993 this was the coolest thing ever.  Even if it did have a reputation of destroying networks.

I can remember how loudly network people would complain about Doom, but it is kind of interesting to see it first hand how chatty it is. But at the same time, how exhilarating it was, for the first time we were playing against each other, not bots.

And speaking of this era, for those who may not have ever read about the rise & fall of iD, you may want to check out the book ‘Masters of Doom‘. Although the end is pretty much Old Man Murray. There is no mention of the soundcard disaster that was DMX, or anything about how deathmatch could crush a LAN. But I guess everything can’t be 100% for the time.

Configuring IPX/SPX

Much like my prior article on Configuring TCP/IP, the process for configuring IPX/SPX on a cisco router is pretty much the same thing.

The first big ‘gottcha’ in the world of IPX is that it supports multiple frame types.  This winds up leading to all kinds of troubles when various people setup various servers.  It is inevitable that people pick their own and incompatible frame type.  The only to ‘fix’ this is for everyone to be on the same type.  I have heard of people using different frame types for different environments, much like using 802.1q (trunking) to separate various peoples traffic who do not want to see each-others resources.  But back when IPX was prevalent, people were still using HUBS which mirror all traffic, and primitive layer two switching at best.  Not to mention with the advent of Netware 4, you could have virtual NIC’s in the server and workstations, and bind to all possible frame types.

Things got messy, and quick.

But I digress, for this example I’m going to use the ‘default’ frame type of ETHERNET_802.2 or SAP is cisco speak.  Mostly because I don’t feel like fully configuring the MS-DOS client I managed to dig up, and mostly because at this point (2013) I really don’t care what frame type I use.

I’m going to setup the following IPX networks:

WAN C0000001
SERVER C0010001
USER C0010002

The first step is to enable the ipx protocol on the router.  This is done with the ‘ipx routing’ command.

corertr1#config t
Enter configuration commands, one per line. End with CNTL/Z.
corertr1(config)#ipx routing
corertr1(config)#exit
corertr1#

Now the IPX protocol is enabled.  The next step is to configure the ethernet interfaces.  This is pretty straightfoward, we put in the network numbers, and assign the correct frame types.

corertr1#config t
Enter configuration commands, one per line. End with CNTL/Z.
corertr1(config)#int eth1/0
corertr1(config-if)#ipx encapsulation SAP
corertr1(config-if)#ipx network c0010001
corertr1(config-if)#exit
corertr1(config)#int eth1/1
corertr1(config-if)#ipx encapsulation SAP
corertr1(config-if)#ipx network c0010002
corertr1(config-if)#exit
corertr1(config)#int fa0/0
corertr1(config-if)#ipx network c0000001
corertr1(config-if)#ipx encapsulation SAP
corertr1(config-if)#exit
corertr1(config)#exit
corertr1#

Now we can quickly check the interfaces that IPX is running on with the ‘show ipx interface brief’ command.  You should get something like this:

corertr1#show ipx interface brief
Interface IPX Network Encapsulation Status IPX State
FastEthernet0/0 C0000001 SAP up [up]
FastEthernet0/1 unassigned not config’d admin down n/a
Ethernet1/0 C0010001 SAP up [up]
Ethernet1/1 C0010002 SAP up [up]
Ethernet1/2 unassigned not config’d admin down n/a
Ethernet1/3 unassigned not config’d admin down n/a
Ethernet1/4 unassigned not config’d admin down n/a
Ethernet1/5 unassigned not config’d admin down n/a
Ethernet1/6 unassigned not config’d admin down n/a
Ethernet1/7 unassigned not config’d admin down n/a

So far, so good.  Now the next question is, does the router see the server?  And what kind of services are available on the network?  This can be found with the ‘sho ipx servers’ command.

corertr1#sho ipx servers
Codes: S – Static, P – Periodic, E – EIGRP, N – NLSP, H – Holddown, + = detail
U – Per-user static
3 Total IPX Servers

Table ordering is based on routing and server info

Type Name Net Address Port Route Hops Itf
P 4 FPNWDC_NW DEAD0001.0000.0000.0001:0451 2/01 1 Et1/0
P 444 VIRTUALLYFUN!FPNWDC DEAD0001.0000.0000.0001:84C8 2/01 1 Et1/0
P 640 FPNWDC DEAD0001.0000.0000.0001:E885 2/01 1 Et1/0

Everything is looking good!  As you can see my virtual Netware server (FPNW on NT 4.0) is type 4, and called FPNWDC_NW.  SAP types 444 and 640 are for Windows NT, with 444 being the NetBIOS Browser/Domain control service.

IPX is a rather chatty protocol, and it is easy for things to go wrong.  Another fun command is ‘show ipx traffic’ which will let you get some idea of what kind of chat is going on.

corertr1#show ipx traffic
System Traffic for 0.0000.0000.0001 System-Name: corertr1
Time since last clear: never
Rcvd: 149 total, 34 format errors, 0 checksum errors, 0 bad hop count,
100 packets pitched, 29 local destination, 0 multicast
Bcast: 126 received, 64 sent
Sent: 79 generated, 18 forwarded
0 encapsulation failed, 2 no route
SAP: 0 Total SAP requests, 0 Total SAP replies, 3 servers
0 SAP general requests received, 4 sent, 0 ignored, 0 replies
0 SAP Get Nearest Server requests, 0 replies
0 SAP Nearest Name requests, 0 replies
0 SAP General Name requests, 0 replies
18 SAP advertisements received, 15 sent, 0 Throttled
4 SAP flash updates sent, 0 SAP format errors
RIP: 2 RIP requests, 0 ignored, 2 RIP replies, 4 routes
9 RIP advertisements received, 30 sent 0 Throttled
7 RIP flash updates sent, 0 atlr sent
4 RIP general requests sent
0 RIP format errors
Echo: Rcvd 0 requests, 0 replies
Sent 10 requests, 0 replies
0 unknown: 0 no socket, 0 filtered, 0 no helper
0 SAPs throttled, freed NDB len 0
Watchdog:
0 packets received, 0 replies spoofed
Queue lengths:
IPX input: 0, SAP 0, RIP 0, GNS 0
SAP throttling length: 0/(no limit), 0 nets pending lost route reply
Delayed process creation: 0
EIGRP: Total received 0, sent 0
Updates received 0, sent 0
Queries received 0, sent 0
Replies received 0, sent 0
SAPs received 0, sent 0
Trace: Rcvd 0 requests, 0 replies
Sent 0 requests, 0 replies

In this brief guide, I’m not going to even get into all of this, however it is important to know that the router has sent things, and received things back.

Another thing that is different about IPX vs TCP/IP is that each server has it’s own internal network, where the services originate from.  You can see this with the show ipx route command

corertr1#show ipx route
Codes: C – Connected primary network, c – Connected secondary network
S – Static, F – Floating static, L – Local (internal), W – IPXWAN
R – RIP, E – EIGRP, N – NLSP, X – External, A – Aggregate
s – seconds, u – uses, U – Per-user static/Unknown, H – Hold-down

4 Total IPX routes. Up to 1 parallel paths and 16 hops allowed.

No default route known.

C C0000001 (SAP), Fa0/0
C C0010001 (SAP), Et1/0
C C0010002 (SAP), Et1/1
R DEAD0001 [02/01] via C0010001.5254.0012.3456, 10s, Et1/0

As you can see from the SAP advertisements above, and the route here, the server is configured for it’s internal network to be DEAD0001.  The router picked up the advertisement from C0010001.5254.0012.3456, which breaks down as c0010001 being the network the server is on (the server network), and 5254.0012.3456 being the MAC address of the server.

The real test comes from trying to use a client.

Screen Shot 2013-10-22 at 1.26.35 PM

Netware 3.12 client

As you can see, this MS-DOS client attached to the client vlan, can see and interoperate with the virtual NetWare server.

This pretty much covers the basics of getting IPX/SPX working.  Logically the next thing to do would be to configure routing and get IPX working throughout the WAN.  But I’m going to save that for a later time.

The more ‘advanced’ topics of IPX involve filtering, as it was a common problem in large networks where you could simply have too many servers, and they would be constantly talking among themselves.  Another problem is licensing, where some products are licensed in certain areas, and you want those licenses to stay in a geographic area.  Latency was another issue too, it was insane to say use a SNA server in Japan, to talk to a mainframe in Arizona when you were in Arizona.  Or if you had a 3rd party, and you only wanted them to connect to a print server, and a single file server, you would setup access-lists on your peer router, much in the same way that we setup firewalls in this fine modern age 😉

Adding some substance to my example network

So thanks to fakenamegenerator.com I thought I should add some people and setup various workstations around my fake network. With that said, here is my list:

name user id country PC
Hazel B. Forrest hforrest USA MS-DOS
James M. White jwhite USA Windows 3.1
Russell I. Ward rward USA
Valerie H. Shimp vshimp USA
Vera H. Williams vwilliams USA
Marie J. Brown mbrown UK Windows 95
Jason S. Seymore jseymore UK
Mingmei Hao mhao HK OS/2 1.21
Guang Huang ghuang HK
Wit Pawlak wpawlak PL WindowsNT 4.0
Fabio Napolitani fnapolitani IT

Which is enough to get me started in creating some users.

For starters I thought it would be fun to make up some applications the users can ‘use’ on this fine network.  A mainframe is a must, however Hercules doesn’t emulate SNA networks.  Which is kind of sad.  I did find an evaluation copy of Microsoft SNA Server 2.11 which runs great on NT 3.5 and higher.  However it is limited to two sessions, but to be honest back when I used a mainframe for work, Microsoft SNA server was honestly the best thing out there.  I had a NT 4.0 / SNA 2.11 install that had uptime in YEARS, while the later SNA 3.0, 4.0 and HIS stuff constantly had issues.

For email, I thought I’d go with something positively ancient, Microsoft Mail 2.1c.

Microsoft Mail

Microsoft Mail

Back then, email programs were just flat databases that allowed multiple people to read/write/lock files over a network. It is very reminisce of how BBS multiuser doors & databases work. MS Mail 2.1 includes clients for MS-DOS, Windows 3.X while the later 3.5 version included an OS/2 client that used the WLO libraries, which was a port of Windows 3.0 to run on top of OS/2.  I kind of covered this thing back here, although it was mostly geared to version 3.5, it basically is the same thing.

I’ve been using a Windows NT 4.0 server loaded up with the FPNW, so it looks like a NetWare server.  Although Netware 3.12 runs on Qemu 0.90, it is lacking UDP bridge support to communicate with dynagen/dynamips.  I did find out that VirtualBOX does support the UDP bridge, and will even run Netware 3.12, HOWEVER, after transferring a few megabytes, the server will stop responding, and dynamips will crash.  Not a very satisfactory solution.  So until I get around to backporting the UDP code, this NT server will serve as my virtual ‘Netware’ server for the time being.

I was also going to run SQL Server 4.21a on WindowsNT, however I did come across SQL Server for OS/2, so I will be installing an OS/2 machine complete with Lan Manager, and SQL Server.  The only downside is that LanManager relies on the non-routable NetBEUI protocol.  However it is just as awkward as bridging mainframe traffic, so I guess that is a hidden plus. While a program to talk to the database outside of the old isql stuff would be nice, I suspect that doing anything beyond Visual Basic + ODBC would take too much time, and honestly not really be all that worth it.

Also looking at this fine program, Stomper, which lets your share a modem over a network, I thought it would be fun to try in combination with rlfossil for some BBSing adventures.  Back before the internet was open to commercial ISPs it wasn’t uncommon for corporations to pool modems over a LAN.  Remote access was typically handled with specialized hardware appliances like the Shiva LanRover. As far as I know, the only real dialup server that Microsoft had was incorporated with Windows NT 3.5.

Once I get this networking operating correctly, then I’ll start to add things like redundancy via HSRP in my core site, backup network connections, an internet connection, upgrade to an exchange server, some BGP peering, and a VPN server.

Just like the real world!