Some history on DPMI

This is taken from a post by Michael Sokolov (of Quasijarus fame), all about DPMI.

   Mark H. Wood <address@hidden> wrote:
> Which spec is that? cwsdpmi, as I recall, implements DPMI 1.0, while
> Microsoft has never gone beyond 0.9. There are significant differences
> *in the spec.s*.

   Well, I have turned this semester's last paper in today, so I can spend
a few more hours enlightening the public... Warning: I am going to give a
lecture on DPMI. Feel free to skip this message, but if you do please
refrain from any further participation in the work on Lynx for DOS, since
an understanding of the material I present in this lecture is absolutely
essential to doing any work under DOS beyond 640 KB.

   DPMI has been born in the fall of 1989 in a bright flash of genius of
Microsoft's Ralph Lipe. At that time there was a flourishing DOS extender
industry led by Phar Lap, Rational Systems (later Tenberry Software), and
Ergo (formerly Eclipse and A. I. Architects). A DOS extender is some kind
of software solution (they come in all sorts of flavors) that allows an
application to execute in protected mode and yet retain access to the file
system and other services provided by DOS via an API that closely resembles
that of real-mode DOS (INT 21h and such) but with pointers to buffers and
such addressing memory outside the first MB via protected-mode selectors.

   DOS extenders from Phar Lap, Rational Systems, and Ergo are inherently
flawed in that they are built into apps. This has been done against all
rationale simply because the market demanded that a protected-mode apps
must not have any prerequisites and must run even in a bare environment
without any pre-existing protected-mode software. However, having a
separate DOS extender for every app is not much different from having a
separate power plant for every light bulb. Since they want to have no
prerequisites, these DOS extenders need to take the total control of the
CPU and build their own protected-mode environments. Of course, this
creates a hell of a problem when a machine IS running some protected-mode
software. In the days of Phar Lap and friends, there were two other kinds
of the latter in addition to DOS extenders: LIMulators and multitaskers.
The former are things like CEMM and EMM386 that provide LIM EMS services by
setting up a protected-mode environment and mapping some extended memory
into the UMB space. The latter provide multiple simultaneous DOS sessions
and allow the user to switch between them instantly. They do so again by
setting up a protected-mode environment and using to it manage several
different VMs. The result was that by 1989 there were three kinds of
protected-mode environments (LIMulators, multitaskers, and DOS extenders),
and all three were, oh big surprise, for the most part mutually exclusive.
There were some feeble attempts (VCPI being the primary example) to make
them live together, but they worked by making one environment totally yield
to the other, which is unacceptable for many reasons.

   Microsoft has entered the protected-mode-under-DOS field in the summer
of 1988 when Murray Sargent and David Weise wrote a DOS extender for a big
DOS app called Windows. Until then this app was running in pure real mode
and was limited by the (in)famous 640 KB barrier. Jumping over this barrier
was one of the primary goals for Windows v3.00. Murray Sargent's DOS
extender did this. It was a quick-and-dirty single-app DOS extender just
like the ones from Phar Lap and others. There was a problem, however. When
all this was happening in the summer of 1988, there already was a baby
called Windows/386 v2.xx. It was another protected-mode environment
developed independently by Ralph Lipe in 1987. It was not a DOS extender,
but a multitasker, and its purpose was to manage several independent VMs,
one running Windows and all others being DOS boxes. Although there were
several VMs, each was running in V86 (real-like) mode only, and no VM had
any ability to jump over the 640 KB barrier, even though each VM had its
own 640 KB of memory.

   Windows v3.00 was supposed to include both Windows/386 (so that it could
have the familiar DOS boxes) and Murray Sargent's DOS extender. There was a
problem, however. The two were mutually exclusive! It was exactly the same
eternal conflict between LIMulators, multitaskers, and DOS extenders that
the industry had been wrestling with for a couple of years by then, except
that the conflicting parties were parts of the same product! Clearly a
breath of fresh air was needed. And it came. From no one but Ralph Lipe,
the author of Windows/386. In a bright flash of genius, he realized that
building a separate DOS extender into every app, be it AutoCAD, Lotus
1-2-3, or Windows, is just as ridiculous as building a separate power plant
for every light bulb. However, getting rid of this kludge required adopting
a new industry standard. Just like in order to have the same power plant
serve everyone you have to agree upon a common standard on voltage,
waveform, and power factor, if you are going to have common protected-mode
DOS services for all apps you have to agree on the protected-mode DOS API.
A logical name for the generic API to the protected-mode DOS was DOS
Protected Mode Interface (DPMI), and it is the name that has ended up being
chosen.

   The specs published by the DPMI Committee and distributed by Intel only
vaguely resemble Ralph Lipe's DPMI. The INT 31h API they describe is only
the low-level building-block portion of Ralph Lipe's DPMI. Its high-level
portion is nothing less than a functional specification for Protected Mode
DOS (hence the name). It allows one to develop an add-on to DOS or even a
new version of DOS that would have a protected-mode API directly usable by
protected-mode DOS apps without the need for separate built-in DOS
extenders. Understandably this troubled Phar Lap and others a lot, since
Ralph Lipe's DPMI makes their products unnecessary. This is the setting in
which the most unfortunate event in the history of DOS has happened. Being
too used to Microsoft's evils, the industry had failed to think the problem
through and sided against Ralph Lipe. Before I continue any further, let me
make an analogy. Suppose a group of doctors were able to cure some dreadful
disease against which there were no vaccine and to which everyone were
prone. Suppose that they charged a lot of money for curing this disease and
made a good living on it. Then suppose that some guy came up with a vaccine
that once injected totally eliminated the possibility of getting this
disease. The vaccine would solve a major health problem, but it would also
push the doctors making great money on curing the disease out of business.
What do you think is more important, sparing millions of people of a major
health problem or keeping a group of lucky doctors in business?

   Unfortunately, the industry has chosen the latter. Ralph Lipe agreed not
to publicize his version of the DPMI spec and to create the so-called DPMI
Committee to write a "public" version of the spec that would please Phar
Lap and the rest of the gang. This Committee was practically headed by Bob
Moote, the vice-president and co-founder of Phar Lap and the author of
their 386|DOS Extender. It has prepared a castrated (deprived of the most
essential component) version of DPMI and released it under the name "DPMI
Version 0.9". It essentially comprises the low-level building-block portion
of True (Ralph Lipe's) DPMI. Fortunately, no committee in the world could
prevent Ralph Lipe from doing the right thing, and the Win386 component in
Windows v3.00+ implements the full True DPMI spec. As a result, DOS apps
running under its supervision (e.g., Windows itself) have immediate access
to Protected Mode DOS without built-in DOS extenders. Since this is the new
proper and vastly improved way of writing protected-mode DOS apps (sure
enough, Microsoft itself has used it in its C/C++ Compiler v7.00), Win386
rightfully does not support apps with built-in DOS extenders using kludges
like VCPI. Thus Bob Moote and his gang have made their version of DPMI a
subset of True DPMI rather than a completely different interface, so that
their DOS extenders could run under Win386.

   It is a very unfortunate fact that since Ralph Lipe has agreed not to
send out copies of his True DPMI spec most people associate the name DPMI
with Bob Moote's "Committee". There were, however, a few copies of the True
DPMI spec sent out to people like Borland. It is interesting to note that
the latter company has chosen True DPMI over Bob Moote's creature for their
protected-mode DOS tools. 16-bit Borland C and Pascal compilers are
targeted for the DOS/protected-mode-DOS/Windows triple, and the protected-
mode DOS versions of the tools use nothing but True DPMI. Also Borland
includes a True DPMI implementation of its own with these compilers. Given
that copies of the True DPMI spec are out there, I'm searching for them
much like Agent Mulder is searching for the truth about extraterrestrial
life and intelligence. Fortunately Bob Moote's "Committee" has never made
DPMI a trademark and Ralph Lipe's spec has never got Microsoft's copyright
stamped on it (it was just a rough dev draft quickly typed up in WinWord,
after all), meaning that if/when I find a copy of it I can immediately put
it up on my FTP site.

   Now you may ask "what is DPMI Version 1.0 then?" As I have said before,
Ralph Lipe and Bob Moote had radically different visions of DPMI, the
former seeking an interface between protected-mode DOS and its apps and the
latter seeking an interface for use by Phar Lap's DOS extenders. "DPMI
Version 0.9" has been produced perhaps in an hour by ripping out Ralph
Lipe's high-level APIs, but there are still references to it dangling
around, making it clear to anyone reading the spec that something is
obviously missing. Being dissatisfied with the fact that his ugly "DPMI
Version 0.9" still contained hints toward its original purpose and didn't
completely subvert to his ideology, he set out to write a spec that would
not contain a single word by Ralph Lipe and instead would be geared fully
toward Phar Lap's DOS extenders. The result is commonly known as "DPMI
Version 1.0". I still wonder why did Bob waste his time on writing it, as
it was clear even to a porcupine that it would never be implemented, since
its alleged purpose was light years apart from the real purpose of DPMI
(which started being accepted by the industry, partly because Microsoft and
Borland compilers used it). It is a grave mistake to treat "DPMI Version
1.0" as an improvement upon "DPMI Version 0.9", it is merely a further move
away from True DPMI.

   A very unfortunate fact is that DJGPP and CWSDPMI have chosen to use Bob
Moote's lopsided idea of DPMI instead of its true nature. Certainly this is
due to DJ Delorie simply not knowing about the existence of True DPMI,
which in turn is certainly due to the scarcity of the copies of the True
DPMI spec. It is still a pity, though, that DJGPP uses only the low-level
building-block APIs that are suitable to Bob Moote's selfish interests and
builds a separate power plant for every light bulb (i.e., builds the DOS
API translation engine into every app by making it a part of the
statically-linked libc.a).

   The conclusion of this lecture is that I need to continue searching for
the truth that is out there, so that I could finally bring it to light
after all these years (True DPMI: 1989, "DPMI Version 0.9": 1990, "DPMI
Version 1.0": 1991). This would mean finding a copy of the True DPMI spec,
putting it up on my FTP site, and writing a software development suite that
would produce apps for it, as well as writing a public-domain
implementation of it (to my knowledge all existing ones are commercial,
except possibly the DOS emulation under Linux).

   A billion thanks for reading! If you read up to here, please drop me a
note :-).

> Stuff compiled with DJGPP works just fine for me on Win95 [...]

   OK, so DJGPP is at least correct after all, although severely misguided
because of the lack of knowledge.

> [...] which is (among
> other things) a DPMI provider and certainly written by Microsoft.

   The Protected Mode DOS component of Windows 95 is virtually identical to
that of Windows v3.00 and v3.1x. In all of them it is written by Ralph Lipe
and is the reference implementation of True DPMI.

> The same stuff also seems just fine using cwsdpmi.

   OTOH, when I tried running a True DPMI client under CWSDPMI, it sent
about 2 KB of binary garbage to stdout and exited. Note that this client
does check for BobMootish DPMI hosts that implement only the low-level
building-block API and prints an appropriate error message if one is found,
so what I have seen indicates that CWSDPMI's implementation of the low-
level building-block API is also screwed. Well, it should be no surprise
for anyone that CWSDPMI is a specialized gadget for DJGPP and doesn't even
resemble True DPMI. If you want the latter without running Windows, your
current choices are (to my knowledge) 386Max and QEMM-386 with QDPMI. There
also is the Borland DPMI host, but it's 16-bit only. Oh, and OS/2, Windows
NT, and Novell DOS all implement True DPMI.

   Sincerely,
   Michael Sokolov
   Phone: 440-449-0299
   ARPA Internet SMTP mail: address@hidden

   P.S. It is really swinish for me to be bashing Bob Moote, since he has
once done me a significant personal favor, but I simply can't put my
personal interests and obligations above my duty to the industry.

   P.P.S. A lot of my knowledge of this subject has been derived from
direct phone conversations with the key players in this field, including
Murray Sargent, Ralph Lipe, Bob Moote, and Dan Spear (author of QEMM-386
and QDPMI).

9 thoughts on “Some history on DPMI

  1. I seriously doubt that the above history is anywhere close to accurate. Microsoft letting some committee dictate what it should and shouldn’t do? On which planet? It’s rather more likely that Microsoft did not want 32-bit DOS, since they needed people to use Windows (3.x and NT).

    And the bit about DOS extenders being bad because they’re built into every app is total nonsense. It completely ignores the history–early DOS extenders had to have everything built in because they simply had no other choice. The companies which could create an extended DOS standard (IBM and Microsoft) were very much not interested in doing so, because it went against their strategic goals.

  2. You sure, it sounds like a page out of embrace, extend & extinguish… The real goal MS wanted was to get people using windows 100% of the time, and the last thing they wanted was a repeat of Windows 2 where people were in & out of Windows. They needed all those extended programs running under Windows (and of course OS/2 2.0 Cruiser!)..

    Although it still had some life in it, there was no doubt that the dos extender market was kind of cooked if Windows could run stuff all by itself so I could see DPMI being a crippled base to keep the extender people in business.

    If Windows could run EXP’s then why would anyone buy Pharlap 386|Dos Extender / TNT.

  3. >Microsoft letting some committee dictate what it should and shouldn’t do?

    Well, according to that article they _didn’t_ let the committe dictate anything.. they just let them go through their motions of defining their standard, and then Microsoft went ahead and implemented their own version (Ralph Lipe’s version) anyway. Business as usual then, the MS way! 🙂

    I wonder if the author found that True Spec.. I’ve not followed the dos extender market since way back before Windows (I never really moved to Windows, any version).

  4. The so-called “True DPMI” is, if I understand it correctly, what IBM calls “DPMI DOS API” in the OS/2 VDM settings. In other words, the INT 21h interface is widened to 32-bit, and the actual DOS extending is unnecessary. However, a 32-bit loader is still needed, as is a 32-bit debugger, and there are are numerous other 16-bit software interrupts which need extending. The upshot is that the “True DPMI” doesn’t actually solve anything even if it’s available.

    Yes, Microsoft didn’t want people to have to get out of Windows, and that’s why they came up with the idea of DPMI. But Microsoft also didn’t really want people to run or write 32-bit DOS applications (or DOS applications at all). Which is why there was no 32-bit application support built into DOS. It would have been easy for Microsoft, but there wasn’t even an optional DPMI server like QEMM and 386MAX (and Novell DOS, I think) had. Which is why Microsoft C/C++ 7.0 had to be shipped with 386MAX just so that DOS users could run it.

    Microsoft only needed a basic interoperability interface, sort of VCPI 2.0. DPMI as it was standardized is actually quite powerful, but not necessarily terribly easy to use. That was part of why DOS extender vendors stayed in business (for a while) — DPMI was not a complete software environment, and I don’t think it was ever meant to be.

    It’s possible that for a while, Ralph Lipe was thinking about a real 32-bit DOS which would directly load and run 32-bit applications, although there’s no solid evidence I’m aware of. It’s pretty clear that Microsoft did not actually want that, because they had a lot more control over Windows than they had over DOS.

  5. “I emailed him, and he never did find the ‘real’ DPMI spec.”
    Even if it was not written up in a real spec, it is not hard to infer if you know what DOS API emulation is.

    • Well that’s funny, because ftp://ifctfvax.harhan.org/pub/micro/msdos/above640k/TrueDPMI/README says that the document you referenced is not the “True DPMI” spec. And that he (Michael Sokolov) never found it.

      The “MS-DOS Extensions for DPMI Hosts” document only confirms what I said. It is not a complete 32-bit DOS environment; one of the biggest missing pieces is the ability to load a 32-bit protected-mode application. In other words, a DOS extender is still needed, even if it does not have to do the job of actually extending the DOS INT 21h API. And because this capability never became part of DOS itself, it’s completely moot — application writers still needed DOS extenders. Unless they thought requiring Windows to run a DOS application was smart. It should also be obvious that however evil and shortsighted the DOS extender vendors were, building “True DPMI” support into DOS was not in their power.

  6. Pingback: Episódio 118 – Windows, os primeiros dez anos – Parte A | Retrocomputaria

Leave a Reply to Yuhong Bao Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.