Chasing more 386 OMF

Microsoft’s first 32bit OS, Xenix

Well back before, I had been looking into old linkers for 386 OMF, I knew I’d found some fun with some old GNU tools going back to the heyday of Xenix 386. As kind of expected the tools used to build Xenix, along with it’s SDK were in fact Microsoft C/MASM. So yes way back in 1987 Microsoft not only has MASM386, but they also had a 32bit Microsoft C 5.00. Let that sink in for a moment as OS/2 had been forced into 16-bit land despite FOOTBALL, and Windows/386 being a 386 VM86 multitasker. So in a weird way all the parts were in existence.

Back in the old days of GCC 1.x there was a bit of excitement about the file masm386.c in the old GCC source directory, sadly despite it being updated, there was no real public push on modifying GCC to support non AT&T assemblers. Instead something unexpected (well to me!) happened is that GAS had been modified to output OMF.

I tested this on MinGW with some simple stuff, and sure enough it links just fine. Considering its what is on the GCC on Xenix port I’m sure it’s pretty solid.

Enter OS/2

Now this is more fun, and again kind of sad that GCC didn’t take on the ability to target other assemblers (just look at the x68000!), Maybe they didn’t see GAS for OMF, or just didn’t know. Instead a more aggressive choice was made, to alter the binary output. Linking on OS/2 with EMX involved 2 very different and incompatible paths, the first one is the ancient Unix i386 a.out format, which then a utility called emxbind will convert into a loader & stub that OS/2 can run. Think of it as an OS/2 extender to take simple Unix programs (which is what they are) to run on OS/2. Neat!

The second more ‘native’ approach is to convert the binary a.out files into what is known as OMF files, which non GNU linkers like Wlink from Watcom or Link386 from Microsoft can then link into direct native EXE’s or DLL’s.

There had been an experimental ELF build of the EMX toolchain on OS/2 but I think it may have died? So as crazy as it seems bigger and crazier programs need to be built on systems like Windows or Linux and linked outside of OS/2 to get around the old memory limits. It’s really hard to say as I’ve never used it, although being able to do the link outside of OS/2 would be an advantage.

I’ve found 2 programs to convert the a.out objects into OMF, the first and oldest being o2obj. The one drawback I’ve had is that this doesn’t play so well with the Watcom 386 OMF linker. Instead the much later RSXNT/LIBC0.6 project’s emxomf. I’ve done some painful hacking but it appears to do what it should do. A simple omfdump seems to be spilling stuff out.

Of course the alternative is to use a 64bit linker, and since a.out has been pulled from binutils the only real hope is the Watcom linker which is now running in a 64bit address space. And the Watcom chain won’t understand ancient i386 a.out, however Microsoft 386 OMF it certainly will, although it appears to be based around something later than the aforementioned o2obj, which is why I ahd to do the emxomf.

I know as this stands it’s not very satisfying but I kind of wanted to push this out the door as I’d been hacking from time to time on it, and didn’t want to leave it to rot completely. The EMX tools remind me of the NeXT stuff where everyone goes native platform wild never imagining a day when remaining portable would mean it’d be easier to target more software.

The one thing I wonder about sometimes if there was some kind of secret Microsoft extended DOS/Windows that relied on OMF & Link386 that predated the NT team and their switch to COFF? Obviously it’d be super obsolete and would have been something like the first PharLap 386 stuff. But I’ve only owned a disk dump of v4, and a legit copy of TNT v6. Old 386/DPMI/Extender tools are hard to find.

Linkers & loaders, along with binary formats are too hard for me, but I thought I’d share at least what I’d been able to conjure with MinGW. I’ll have to touch on EMX to native OMF linking later.

2ine the OS/2 emulator

So this is really super cool! Ryan C. Gordon has written a Wine like program to run OS/2 programs!

Using 32bit Linux, and some native libraries, 2ine can load up an LX (32bit) executable and try to run it under Linux, much in the same way that Wine can run Windows programs.  And yes it’ll run EMX built stuff.  Although keep in mind the original Microsoft based languages, programs and tools is all 16bit.  After the whole Windows 3.0 thing and the split of Microsoft from the OS/2 project all their tools are either 16 bit, or 32bit LE format, which IBM had dumped for the LX format once OS/2 2.0 had shipped.

You can read about his incredible progress, and all the trials and tribulations of running OS/2 programs, along with the craziness that is thunking back and forth to the 16bit space for the old VIO calls that had never were updated to 32bit in that transition phase where a good chunk of OS/2 never was updated from 16bit, over on his patreon page here.

Attempting to run anything 16bit or LE will give you:

./lx_loader CL.EXE
not an OS/2 LX module

But let’s try my crazy Win32 hosted EMX 0.8h cross compiler!

C:\emx\demo\dhry>gcc -v dhyrstone.c -o dhyrstone.exe
gcc version 2.5.8
cpp -lang-c -v -undef -D__GNUC__=2 -D__GNUC_MINOR__=5 -D__32BIT__ -D__EMX__ -Di386 -D__32BIT__ -D__EMX__ -D__i386__ -D__32BIT__ -D__EMX__ -D__i386 -Asystem(unix) -Asystem(emx) -Acpu(i386) -Amachine(i386) dhyrstone.c C:\Temp/cca13032
GNU CPP version 2.5.8 (80386, BSD syntax)
#include “…” search starts here:
#include <…> search starts here:
/usr/local/include
D:/pcem/building/MinGW/msys/1.0/local/emx/include
/emx/include
/usr/include
End of search list.
cc1 C:\Temp/cca13032 -quiet -dumpbase dhyrstone.c -version -o C:\Temp/ccb13032
GNU C version 2.5.8 (80386, BSD syntax) compiled by GNU C version 5.1.0.
as -o C:\Temp/ccc13032 C:\Temp/ccb13032
ld -o dhyrstone.exe /emx/lib/crt0.o -L/emx/lib C:\Temp/ccc13032 -lgcc -lc -lgcc -lemxst -los2 -lemx2

And now running that on Linux…

root@alpharacks:/usr/src/2ine-4a8318f4056f# file dhyrstone.exe
dhyrstone.exe: MS-DOS executable, LX for OS/2 (console) i80386, emx 0.8c
root@alpharacks:/usr/src/2ine-4a8318f4056f# ./lx_loader dhyrstone.exe
Dhrystone(1.1) time for 5000000 passes = 3
This machine benchmarks at 1666666 dhrystones/second

You’d never know that this was an OS/2 program, if I didn’t tell you.

I tried the old 87 Infocom interpreter, and it’ll run great too!

root@alpharacks:/usr/src/2ine-4a8318f4056f# file infocom.exe
infocom.exe: MS-DOS executable, LX for OS/2 (console) i80386, emx 0.8c

root@alpharacks:/usr/src/2ine-4a8318f4056f# ./lx_loader infocom.exe advent.z3

At End Of Road Score: 36/0
Welcome to Adventure! Do you need instructions? (y/n) >n

ADVENTURE
A Modern Classic
Based on Adventure by Willie Crowther and Don Woods (1977)
And prior adaptations by David M. Baggett (1993), Graham Nelson (1994), and
others
Adapted once more by Jesse McGrew (2015)
Release 1 / Serial number 151001 / ZILF 0.7 lib J3

At End Of Road
You are standing at the end of a road before a small brick building. Around you
is a forest. A small stream flows out of the building and down a gully.

At End Of Road Score: 36/0
>

Again it’s works so well it’s amazing!

You can find the 2ine source over on icculus.org here.  I had to tweek the heck out of the CmakeList.txt to get it to build, and since I was interested in the command line, I ended up disabling all the SDL / PM stuff, and make sure I had the ‘wide/unicode’ version of ncurses installed.

I don’t think there really was any killer 32 bit OS/2 applications, but with clean room versions of:

  • doscalls.dll
  • kbdcalls.dll
  • msg.dll
  • nls.dll
  • quecalls.dll
  • sesmgr.dll
  • tcpip32.dll
  • viocalls.dll

Not to mention being able to call into Linux DLL’s and using ‘clean’ OS/2 DLL’s would let you embrace and extend OS/2.. Or maybe even let you build the proverbial fantasy of both RISC & 64 bit OS/2. …..

Found EMX 0.8b

While cruising archive.org, I found this CD-ROM image, “OS/2 Archive CD-ROM Walnut Creek May 1992“, which included the following zoo files:

05/05/1992 09:46 AM 144,272 EMXDEV.ZOO
05/05/1992 09:44 AM 167,809 EMXINFO.ZOO
05/05/1992 09:46 AM 101,132 EMXLIB.ZOO
05/05/1992 09:46 AM 19,266 EMXTEST.ZOO
05/05/1992 09:46 AM 586,285 GNUDEV.ZOO
05/05/1992 09:40 AM 78,575 GNUPAT.ZOO
05/05/1992 09:41 AM 138,891 GNUSRC1.ZOO
05/05/1992 09:41 AM 184,671 GNUSRC2.ZOO
05/05/1992 09:45 AM 1,044,875 GNUSRC3.ZIP
05/05/1992 09:43 AM 1,015,692 GNUSRC3.ZOO
05/05/1992 09:42 AM 505,127 GNUSRC4.ZOO
05/05/1992 09:45 AM 3,178 README.DOC

And from the readme, the release is from Feburary of 1992.  Keeping in mind the GA release of OS/2 2.0 was released in April of 1992.

EMX 0.8b INTRODUCTION 22-Feb-1992

Welcome to emx, a common environment for creating 32-bit programs for OS/2 2.0
and MS-DOS. You can use the GNU C compiler to compile programs for EMX.

Included in the emx package are:

  • emx.dll dynamic link library for OS/2 2.0
  • emx.exe DOS extender for running 32-bit programs under MS-DOS
  • emxbind.exe for creating .exe files which work both under OS/2 2.0 and
    MS-DOS
  • C header files and a nearly complete C library, including source

Additionally, the following GNU programs are available compiled and with
sources (note that these files are not part of EMX):

  • gcc, the GNU C compiler
  • gas, the GNU assembler
  • gdb, the GNU debugger
  • ld, the GNU linker
  • ar, nm, size, strip, objdump: some GNU utilities for dealing with binary
    files
  • Patched source for gcc, gas, gdb, ld, ar, nm, size, strip, objdump. You can
    compile all these programs with the files that come with emx (but you also
    need a make utility, such as NMAKE)

So this pretty much sums it up.  I went ahead and extracted the ZOOs and placed a copy on my site: emx08b_extracted.7z  Although I don’t think anyone really cares about ancient versions of GCC on OS/2.

EMX 0.9d rehosted on Win32

EMX on Win32

I know it’s utterly pointless… But yeah GCC 2.8.1 + EMX 0.9d, hosted (running) on Win32.  The main reason is that I wanted to be able use use my substantially faster Win64 machines to build stuff for OS/2.  And since I have a 4 core (+4 hyper thread), I want to be able to use make with the -j 16 flag, and say compile QuakeWorld/2 in under two seconds.

I was able to get the binutils 2.6 derived stuff to compile, along with the ‘ancient’ binutils which is notably the linker that EMX depends on.  I would imagine this ought to be able to compile PDOS, although my own simple attempt at InfoTaskForce met with spectacular failure.  While it does compile fine using an older EMX 0.8h based release.

EMX 0.9d on Windows 10 x64

EMX 0.9d on Windows 10 x64

As you can see, it can compile the dhyrstone benchmark, and run the MS-DOS version via the MS-DOS Player.

Downloads

EMX GCC 2.5.8 for Windows

Continuing in my quest of useless stuff, here is GCC 2.5.8 for EMX as a native Win32 executable set.

With some weirdness of inline not being recognized, I just commented that out.  The GCC driver program does have some issues though, and Im just not in the mood to fully figure out why either the EMX or MS-DOS versions can’t correctly  capture return codes, or if it was better to just use -pipe and _pipe with _O_TEXT mode set, but again how to figure out if the pipe closed cleanly or with errors?  So for now it’ll always assume everything worked, but it will still print errors.  Sigh.

With that said, the CC1, CC1OBJ, CC1PLUS drivers all built, so you can use C, ObjectiveC, and C++. yay.

The functional version of this is EMX 0.8H, with the gcc 2.5.8 update.  Maybe I’ll put this all on sourceforge so people don’t have to deal with my crappy download system, but for now it’s on my site.

Here is my build, along with binaries:

emx_gcc_258_phase1_xgcc_cccp_cc1_cc1obj_cc1plus.7z

And the un touched source code, as provided in the 0.8H update

emx_gcc_258_virgin.7z

I’ve only tested it with MS-DOS, and PDOS.  I’m also using the ancient binutils from my GCC 1.40 on Windows experiment.

GCC 1.40 on Windows

I know with all the talk of GCC 6.1.0 for MS-DOS, and other platforms, you must be thinking that all this talk of progress, and high versions numbers just isn’t right!  I’ve just started to migrate code to GCC 5.1, and now you are telling me there is a GCC 6!

Where can I turn away from all this so called progress!  I don’t like my C compilers to be C++ programs that require massive HOURS to compile.  Can’t we just go back to the good old days?

And the answer is YES, you can!

While looking for some libraries on another project, I came across this old defunct project called RSXNT. And it’s a port of EMX to Win32 platforms!  Well isn’t that fantastic!

So, considering I was able to build GCC 1.40 and cross compile to Linux 0.11 from Windows, can we do something with this?

Well ancient versions of EMX are very difficult to track down.  Somehow I did mange to find this hybrid of 0.8B & 0.8C.  The EMX runtime & binaries are from 0.8C, but the source code is from 0.8B.  And the best thing is that the 0.8B is based around GCC 1.40!  So with a little bit of tweaking the files, and messing around I got the assembler, linker, and C compiler to build with MinGW!  Sadly the source code to EMXBIND, wasn’t included in the zips that I have, but the aformentioned RSXNT packages included a version of EMXBIND that will run on Windows!  So I managed to mash them together, and for the fun of it, I’m using the old InfoTaskForce interpreter from 1987 to complete the vintage feel.

Compiling & Binding

Compiling & Binding

Now with my executable, I can run it on MS-DOS & OS/2!

MS-DOS via DOSBox

MS-DOS via DOSBox

and OS/2 2.0!

OS/2 (on Qemu)

OS/2 (on Qemu)

Well isn’t that fantastic!

However when running RSXNT’s bind, NTBIND I got this error:

D:\emx_w32\infocom>..\bin\ntbind info2
No relocations in file:
you have not linked the NT library

Great.  Some more digging around, and if you want to make Windows programs, you need to use the RSXNT includes & libraries.  So I shifted the libraries around, and patched gcc to call the linker the same way RSXNT’s gcc driver calls it, and first go this error:

io.o: Undefined symbol __streams referenced from text segment

And looking at the stdio.h there is this:

extern struct _stdio _streams[];

No doubt, the headers & libraries are tied together.  So now making both of the RSXNT versions, I can link the executable. (YES I did try declaring the structure anyways, and I get stdout, but stdin doesn’t work).

Running on Windows 10

Running on Windows 10

Just like EMX before it, RSXNT, requires you to have the RSXNT.DLL file in your path, or in the same directory.  I suppose it’s a fair trade off.  Not that I expect there to be a surge of people cross compiling from Windows to OS/2, or even MS-DOS these days.  GCC 1.40 is ancient, 1991 vintage, but even Linus Torvalds loved it!

For comparison, GCC 5.10 produces a 55,906 byte interpreter, while GCC 1.40 produces a 88,576 byte interpreter.

For an attempt at porting some code, I choose Nethack 1.3d, and used the MS-DOS based makefiles.  It didn’t work so well, but I was able to patch in enough of the unix based termios logic, and thanks to EMX/RSXNT’s built in termios capabilities I was able to get a working version!

Nethack 1.3d on Windows 10 x64

Nethack 1.3d on Windows 10 x64

I don’t know if there really was any advantage to compiling with GCC 1.40, but it was great to see that this 1991 compiler could handily compile the 1987 based code.

How about some speed comparisons?  I dug out the ancient dhrystone.c, and gave it a shot.  I had to define 500,000,000 passes, as my computer is fast.  GCC 1.40 only offers -O for optimization, while GCC 5.1 offers many more levels, but for this quick experiment they really aren’t needed.

D:\emx\demo\dhry>gcc140.exe
Dhrystone(1.1) time for 500000000 passes = 57
This machine benchmarks at 8771929 dhrystones/second

D:\emx\demo\dhry>gcc140_O.exe
Dhrystone(1.1) time for 500000000 passes = 40
This machine benchmarks at 12500000 dhrystones/second

D:\emx\demo\dhry>gcc510.exe
Dhrystone(1.1) time for 500000000 passes = 43
This machine benchmarks at 11627906 dhrystones/second

D:\emx\demo\dhry>gcc510_O.exe
Dhrystone(1.1) time for 500000000 passes = 16
This machine benchmarks at 31250000 dhrystones/second

D:\emx\demo\dhry>gcc510_O2.exe
Dhrystone(1.1) time for 500000000 passes = 14
This machine benchmarks at 35714285 dhrystones/second

D:\emx\demo\dhry>gcc510_Ofast.exe
Dhrystone(1.1) time for 500000000 passes = 11
This machine benchmarks at 45454545 dhrystones/second

As you can see, GCC 1.40 produces the slowest code.  While it’s optimized code did beat out GCC 5.10 with no optimizations, turning on optimizations did blow it away.  And again GCC 5.1 beat out the older 1.40 for executable sizes.

29,960 gcc510_O.exe
29,996 gcc510_O2.exe
30,472 gcc510.exe
70,656 gcc140_O.exe
74,752 gcc140.exe

And this time by over a 2x lead!  It is fair to say that the new versions of GCC, despite being significantly larger do indeed produce smaller and faster code.

For anyone who’s read this far, I guess you want to take it out for a test drive?  Remember it is still EMX based, which means is wants to live on the ROOT of your hard disk.  I’m using the ‘D’ drive for myself, so if you are using C or whatever you’ll need to alter the environment vars.

You can download the exe’s and combined source here: gcc-1.40_EMX-OS2_RSXNT.7z

QuakeWorld for OS/2 2.0

Not sure why its suddenly working…. but I suspect it may be either updates to both OS/2 base OS & TCP/IP or…. it is because I’m using the QuakeWorld server code that matches the client…. Anyways I’ll upload a binary and the rest later as it is super late.  But for those of you who want to see it…

Yes it really is an OS/2 exe built with EMX!

I’ve updated the sourceforge page to include an exe, and a copy of the updates that I’m using to OS/2.. Oddly enough my OS/2 install with Virtual PC no longer works… The NIC isn’t found anymore, must be some update?  I’ve got 6.0.192.0, although I know for a fact that this image used to work…

Broken driver

Further update, turns out I’m retarded the AMD PCNet driver is for VMWare/Qemu … Virtual PC emulates a DEC 21140a, which I downloaded a NDIS2 driver from Intel which works great.  I do have to turn off hardware assisted virtualization otherwise OS/2 won’t boot at all from the hard disk..  I’m not sure if it is because I’m now on an AMD computer, or if it is the matched QuakeWorld server/client but it works fine… in Qemu & Virtual PC.

Failed Quakeworld port to OS/2 2.0

Yeah, I guess I just couldn’t salvage this thing… Sometimes when you stand on the shoulders of giants you fall off.

So what had started as a seemingly simple thing turned into a nightmare.. I took the source to my MS-DOS port of Quakeworld, and decided to build it under OS/2. And to be crazy about it, I thought it’d be awesome to get it to work under OS/2 2.0 .

Which means no sound, locked to 320×200 256 colours, full screen only, since the VGA driver won’t do anything but. Years ago on ebay I managed to score IBM TCP/IP for OS/2 2.0 & 2.1, with the LAN support!

IBM OS2 2.0 syslevel 1

IBM OS2 2.0 XR02000 (no fixpacks)

 

 

ibm os2 2.0 syslevel2

IBM TCP/IP for OS/2 with no fixpack

So as you can see from my syslevel’s I’m running a pretty bare machine, even the graphics subsystem is still 16bit!

Well the good news, is that with EMX and a HPFS disk, I was able to quickly get the null version running. With a minor amount of work, I had TCP/IP connectivity and things were going well. I had ripped apart a video demo, and got the display up, although it was ungodly slow.. Until I found out that when you are full screen you can request access to the physical video memory, under OS/2 so that sped things up a great deal!

Then all that really needed to be done was get the keyboard working.

And here is where I slipped up.

OS/2 is weird for interactive programs that want to know key states. It seems the best and most common way to do this was to setup a ‘monitor’ for the keyboard device, and try to read the stream as keys are pressed. Sadly idiots that slamm the keyboard, or gamers that hold down an arrow key for a minute lock the thing, and the key then is ‘jammed’ down until you hit it again for the release.

I’m pretty sure that I did it wrong, so I borrowed code from a more ‘advanced’ OS/2 port and it did the same thing. Maybe it’s OS/2 2.0? I don’t know, I did service pack it and update and still the same result.

I really didn’t want to spend that much time on it…..

QuakeWorld for OS/2

QuakeWorld for OS/2

I feel hesitant to release a binary as you will get killed, and I haven’t even tried the mouse yet, but I suspect it’ll still suck.

So for anyone that cares, here is the source… misspellings and all.