Apparently talking about DOS Extenders is too hot for Twitter: AKA Phar Lap 386

I had a small twitter account, and I tried not to get dragged into anything that would just be basically wasting my time. Just stay focused and on topic. FINE. I just wanted to see if anyone ever saw it, if it was even worth the effort of doing WIP’s as I didn’t want to make it super annoying.

I logged on to post a fun update that I’d finally gotten a Phar Lap 386 version 4.1 app to do something halfway useful, the sairen AGI interpreter up and running in the most basic sense.

Talking about DOS Extenders is spammy and manipulation!

I don’t get what triggered it, but oh well there was a ‘have a review’ and yeah that was fine. Great. So I’m unlocked so I go ahead and post with the forbidden topic, as I’m clearly dumb, and forgetting that Twitter is for hate mobs & posting pictures of food, and cat pictures.

The Sairen AGI interpreter built with Watcom 386/7.0 & Phar Lap 386 4.1

So yes, that was a line too far, and now that’s it.

Now some of you may think, if you buy ‘the plan’ you’ll no doubt be exempt from the heavy hands of Twitter

3 squids a month

But I already was and had been for a while.

Your account is suspended

So that’s the end of that. I guess it’s all too confusing for a boomer like me.

Cancel me, cancel you

So needless to say I cancelled Twitter as well. Kind of sneaky they didn’t auto-cancel taking money.

So yeah, with that out of the way, let’s continue into DOS Extender land. I added just enough 386 magic, onto github: neozeed/sarien286. Yes I see now it really was a poorly named repo. Such is life.

There is 3 main things for porting old programs where they take care of all the logic, it’s going to be File I/O, Screen I/O, and timers. Luckily this time it was easier than I recalled.

Over on usenet (google groups link) Chris Giese shared this great summary on direct memory access from various methods:

/* 32-bit Watcom C with CauseWay DOS extender */
int main(void) {
char *screen = (char *)0xA0000;
initMode13();
*screen = 1;
return 0;
}

/* 32-bit Watcom C with DOS/4GW extender
(*** This code is untested ***) */
int main(void) {
char *screen = (char *)0xA0000;
initMode13();
*screen = 1;
return 0;
}

/* 32-bit Watcom C with PharLap DOS extender
(*** This code is untested ***) */
#include <dos.h> /* MK_FP() */
#define PHARLAP_CONVMEM_SEL 0x34
int main(void) {
char far *screen = (char far *)MK_FP(PHARLAP_CONVMEM_SEL, 0xA0000);
initMode13();
*screen = 1;
return 0;
}

/* 16-bit Watcom C (real mode) */
#include <dos.h> /* MK_FP() */
int main(void) {
char far *screen = (char far *)MK_FP(0xA000, 0);
initMode13();
*screen = 1;
return 0;
}

It is missing the Phar Lap 286 method:

/* Get PM pointer to text screen */
  DosMapRealSeg(0xb800,4000,&rseg);
  textptr=MAKEP(rseg,0);

But it’s very useful to have around as documentation is scarce.

Which brings me to this (again?)

Phar Lap 386|Dos-Extender 4.1

Years ago, I had managed to score a documentation set, and a CD-ROM with a burnt installed copy of the extender. I didn’t know if it was complete, but of course these things are so incredibly rare I jumped on the chance to get it!

2011!

Unfortunately, I didn’t feel right breaking the books apart, and scanning them, then add in some bad life choices on my part, and I ended up losing the books. Fast forward *years* later and Foone uploaded a document set on archive.org. GREAT! As far as I can tell the only difference in what I had is that I’ve got a different serial number. Thankfully I was smart enough to at lest email myself a copy of the CD-ROM contents! And this whole thing did inspire me to gut and upload the Phar Lap TNT 6.0 that I had also managed to acquire.

Although unlocking the video RAM wasn’t too bad, once I knew what to do, the other thing is to hook the clock for a timer. ISR’s are always hell, but at least this is a very simple one:

void (__interrupt __far *prev_int_irq0)();
void __interrupt __far timer_rtn();
int clock_ticks;
#define IRQ0 0x08
void main()
  {
   clock_ticks=0;
   //get prior IRQ routine
   prev_int_irq0 = _dos_getvect( IRQ0 );
   //hook in new protected mode ISR
   _dos_setvect( IRQ0, timer_rtn );

/* do something interesting */
   //restore prior ISR
   _dos_setvect( IRQ0, prev_int_irq0 );
  }

void __interrupt __far timer_rtn()
  {
    ++clock_ticks;
    //call prior ISR
    _chain_intr( prev_int_irq0 );
  }

The methodology is almost always the same, as always, it’s the particular incantation.

So yeah, it’s super simple, but the 8086/80286 calling down to DOS/BIOS from protected mode via the int86 just had to be changed to int386, and some of the register structs being redefined. I’m not sure why but the video/isr code compiled with version 7 of Watcom, but crashes. I think its more drift in the headers, as the findfirst/findnext/assert calls are lacking from Watcom 7, so I just cheated and linked with Watcom 10. This led to another strange thing where the stdio _iob structure was undefined. In Watcom 10 it became __iob, so I just updated the 7 headers, and that actually worked. I had to include some of the findfirst/next structures into the fileglob.c file but it now builds and links fine.

Another thing to do differently when using Watcom 7, is that it doesn’t include a linker, rather you need to use 386LINK. Generating the response file, as there is so many objects didn’t turn out too hard once I realized that by default everything is treated as an object.

Another fun thing is that you can tell the linker to use the program ‘stub386.exe’ so that it will run ‘run386’ on it’s own, making your program feel more standalone. From the documentation:

386 | LINK has the ability to bind the stub loader program, STUB386.EXE, to 
the front of an application .EXP file. The resulting .EXE file can be run by 
typing the file name, just like a real mode DOS program. The stub loader 
program searches the execution PATH for RUN386.EXE (the 

386 | DOS-Extender executable) and loads it; 386 | DOS-Extender then loads 
the application .EXP file following the stub loader in the bound .EXE file. 


To autobind STUB386.EXE to an application .EXP file and create a bound 
executable, specify STUB386.EXE as one of the input object files on the 
command line.

So that means I can just use the following as my linker response file.

agi.obj,agi_v2.obj,agi_v3.obj,checks.obj,cli.obj,console.obj,cycle.obj
daudio.obj,fileglob.obj,font.obj,getopt.obj,getopt1.obj,global.obj
graphics.obj,id.obj,inv.obj,keyboard.obj,logic.obj,lzw.obj,main.obj
menu.obj,motion.obj,pharcga3.obj,objects.obj,op_cmd.obj,op_dbg.obj
op_test.obj,patches.obj,path.obj,picture.obj,rand.obj,savegame.obj
silent.obj,sound.obj,sprite.obj,text.obj,view.obj
words.obj,picview.obj stub386.exe
-exe 386.exe
-lib \wat10\lib386\dos\clib3s.lib \wat10\lib386\math387s.lib
-lib \wat10\lib386\dos\emu387.lib

It really was that simple. I have to say it’s almost shocking how well this went.

So, this brings me back, full circle to where it started, me getting banned for posting this:

32bit!

I thought it was exciting!

For anyone who feels like trying it, I prepped a 5 1/4″ floppy disk image.

running on 86box, 386DX-40 CGA graphics

One interesting observation is that the 386 extender is actually smaller than the 286 one. And being able to compile with full optimisations it is significantly faster.

16bit on the left, 32bit on the right.

I ran both the prior 16bit protected mode version (on the left), and 32bit version (on the right), on the same IBM PS/2 80386DX 16Mhz machine. You can see how the 32bit version is significantly faster!.

I really should profile the code, and have it load all the resources into RAM, it does seem to be loading and unloading stuff, which considering were in protected mode, we should use all ram, or push the VMM386 subsystem to page, and not do direct file swapping, like it’s the 1970s.

Check out Julio Merino’s article on real mode memory allocation:

From 0 to 1 MB in DOS

https://blogsystem5.substack.com/p/from-0-to-1-mb-in-dos

I’m looking forward to how the series progresses, even DJ has dropped on in on the article!

So much trauma, 64kb segments, wrapping around, a20 gates, and overlaid virtual machines. And that’s not even really going into extenders. Looking forward to segmentation vs paging, interrupt reflection, and bi-modal handlers! Don’t cross the ISA 16Mbit DMA boundary!

Happy 2024

Sorry I’d been working on something but it’s just not going to happen quick enough for some NYE surprise.

1/1/2024

Some of my favorite highlights include:

With of course the totally unexpected surprise of ‘Windows 2000 64-bit for Alpha AXP‘. What a year!

Hopefully the new year is a prosperous one.

Surprise Firmware update & BitLocker

So, I rebooted my Windows laptop after some update, and was greeted by this surprise!

well that was a surprise

I guess lucky for me I had the machine tied to a Microsoft account which in turn had the key backed up. After some tedious typing later and I was back in business.

I guess this was the culprit?

BitLocker was so transparently well working I didn’t even know it was even running. Got to give it that much credit. And if anything, probably should just give up and run it everywhere.

It’s really awkward when your mistakes are public

Blessed is the RS232

Blessed is the RS232

On the way back from a day trip with my muse I came across this interesting thing on all the sign boards in Waverly. Not sure if it’s a firmware update that went rogue, or if someone is messing with them, but for the most part all the signs would alternate between the two. I guess if I was running the rails I too would have them set on a serial MUX and keep that crap off a network to make the bar higher for the haxor kids.

But all the same, if someone goofed up, and zapped all the boards, they better set their serial lines correctly to get ahold of them. And maybe update from V14.5, I mean 2017 is so old!

Expect more DEC Alpha madness coming in. And as always, I want to thank my Patreon‘s for making all of this crazyness possible!

#include <names.h>
Brian Ledbetter
Joakim L. Gilje
Bamdad
Andy Wallis
Endir
Tink

Extracting warc files

well since the catastrauphic outage, I’ve been looking through my backups trying to see how much of my ‘vpsland’ archive I have. And it’s not so hot. The good news is the physical machine that has the last known good copy is fine. It’s just in a place I can’t get to on the other side of the world. And I’m still in exile so shipping it really isn’t an option at the moment.

On the plus side I found a warc archive, some 22GB of the 400GB worth of files. So its a start.

So what are WARC files? why do people gzip them to get maybe 1% compression? How do magnets work anyways?

web archives are single snapshots in time of a site. Sounds like a MHT but something more ‘portable’ and open standard-ish. Which means there is a million tools, none of which seem to do exactly what you want.

All I want to do is extract all my files from the WARC, but that seems to not be what most things are geared to, mostly displaying the WARC like a web page, which means clicking hundreds of thousands of files. –yikes

Thankfully warcat seems to be able to fit the bill

python3 -m warcat extract ../user-giddythrill28@vpsland.superglobalmegacorp.com-old-install-2015-10-04-fc233ad0-00000.warc.gz

I didn’t see any package on Ubuntu so did the pip install:

pip3 install warcat

And that seems to have done the trick.

Now to figure out how to setup some cheap storage on azure and copy this stuff up or extract over there.

spot pricing

I’m using the new ‘spot‘ pricing model, to try to keep costs down. Obviously it’s not as good as dedicated slices, but it’ll not make me broke either. And I have a lot more messing around with containers to do, trying to string together nonsense.

Launching Patreon

So yeah, it wasn’t anywhere near as tedious as I thought it would be.

https://www.patreon.com/virtuallyfun

I’ve set prices in Hong Kong dollars, but it claims it’ll re-adjust to your local currency, so it shouldn’t look too crazy as $40 HKD is about $5 USD. After they take all their fees and stuff I get like half. Which I’m not complaining, a week ago it was a big goose egg!

I’ve tried to make the tiers a little fun sounding, although with of course the most important thing we’ve learned on this fun trip is that you absolutely need either an American keyboard, or an Apple branded keyboard to enter recovery mode on a Macintosh while abroad.

As always everything will eventually wind up here. I’ll do some additional collaboration over on Patreon to give it ‘value’ even if it’s just me hitting my head against the wall.

I hate shilling for this kind of thing, ring the bell and click the thing. I’m really embarrassed that it’s come to that here, but like the ads I really don’t have much of a choice at the moment. And of course your direct support by either enabling ads on this page, or signing up just makes it all the easier to keep on through these times, as I’m aware that we are all going through some real fun challenges at the moment.

So a very big preemptive thank you very much, I’m always amazed that whenever I write anything that people read it, and I’m always shocked and amazed to get not only any kind of engagement, but an actual following. Just as I’ve been amazed to have already met so many of you in my travels around the world, through the years!

I would like very much to build on this into bigger things, create more diverse and engaging content, and have some fun doing so!

A big thank-you!

Jason

How to fix rsync slowing down over time (SOLVED)

(This is a guest post by Antoni Sawicki aka Tenox)

I often make copies of large data archives, typically many TB in size. I found that rsync transfer speed slows down over time, typically after a few GB, especially when copying large files. Eventually reaching crawl speeds of just few KB/s. The internet is littered with people asking the same question or why rsync is slow in general. There really isn’t a good answer out there, so I hope this may help.

After doing some quick profiling I found out that the main culprit was rsync's advanced delta transfer algorithm. The algorithm is super awesome for incremental updates as it will only transfer changed parts of a file instead of the whole thing. However when performing initial copy it’s not only unnecessary but gets in the way and the CPU is spinning calculating CRC on chunks that never could have changed. As such…

Initial rsync copies should be performed with -W option, for example:

$ rsync -avPW <src> <dst>

The -W or --whole-file option instructs rsync to perform full file copies and do not use delta transfer algorithm. In result there is no CRC calculation involved and maximum transfer speeds can be easily achieved.

Long term, rsync could be patched to do a full file transfer if the file doesn’t exist in destination.

Also while copying jumbo archives of many TB I don’t want to see every individual file being copied. Instead I want a percentage of the total archive size and current transfer speed in MB/s. After some experiments I arrived at this weird combo:

$ rsync -aW --no-i-r --info=progress2 --info=name0 <src> <dst>