Fun with Apache, (mod_proxy, mod_rewrite), stunnel, And AltaVista Personal search

As you may remember from my prior attempt at using Altavista Search I ran out of space, and found out it only serves pages on 127.0.0.1:6688 and is pretty much hardcoded to do so.  It’s a “fine” hyrid java 1.01 application, with the bulk of it being java.  I finally got around to setting up a VM, and unpacking all of the utzoo archives, and indexing them.  I should have done something about the IO because this took too long (KVM).

SIXTEEN HOURS!!!

SIXTEEN HOURS!!!

So to cheat the system, I installed stunnel as a simple https to http proxy, which let me access my search VM anywhere.  However it still embedded 127.0.0.1 in all the pages.

via stunnel

via stunnel

Enter an Apache reverse proxy to talk to stunnel to talk to AltaVista search!

First to enable a few modules:

a2enmod substitute
a2enmod proxy
a2enmod ssl
a2enmod proxy_http
a2enmod rewrite

And adding this into the config:

SSLProxyEngine On
ProxyPass “/altavista/” “https://10.12.0.16”
ProxyPassReverse “/altavista/” “https://10.12.0.16/”
ProxyRequests Off
RewriteEngine On
SetOutputFilter INFLATE;SUBSTITUTE;DEFLATE
AddOutputFilterByType SUBSTITUTE text/html
Substitute “s/1997/2016/ni”
Substitute “s/97/16/ni”
Substitute “s|127.0.0.1:6688|debian7/altavista|n”
Substitute “s|file:///C:\Program Files\DIGITAL\AltaVista Search\My Computer\images\|http://debian7/images/|n”
Substitute “s|launch=app||n”
Substitute “s|<a href=http://debian7/altavista/?pg=q&what=0&fmt=d|<!—|n”
Substitute “s|><strong>|—>|n”
Substitute “s|</strong></a>||n”
Substitute “s|>u:\|->u:\|n”

This let me redirect all of those requests into a VM called debian7 on the /altavista path.  I also copied the images to the apache server, and now I get something that looks correct!

Apache in the mix!

Apache in the mix!

I cut the results short… But here is a search of something simple:

About 16598 documents match your query.

About 16598 documents match your query.

I also killed all the ‘working URL’s that simply open a desktop application on the index ‘server’.  Naturally it was a personal service, but as a server this isn’t any good.  As such you can’t click on any search results now.  I need something else to figure out how to take the result blocks like “u:\b128\comp\databases\2852” and turn them into URL’s.

Also, as much as I want to re-index I would be best to cut off the headers, or most of them so the preview lines make sense.  Xref, Path, even From & Newsgroups don’t interest me.

I hate to leave it as ‘good enough’ but if anyone has a solution…. I’ll be glad to make this wonderful resource available!

Installing Microsoft Java on Windows 2000 SP4

So yeah, I’m using some old crap software, and it wants MS Java. Great. Ive installed Windows 2000 + SP4, IE6, and then to install MS Java and I get this error:

protected system component?

protected system component?

The Microsoft VM you are attempting to install is a protected system component and can only be updated with a later release of the operating system or service pack.

Well as far as I know, msjavx86-5.0.3810.0 is the last release of MS Java, so what to do?

Apparently all you have to do is rename it to ‘msjavwu.exe’ and it’ll install.

MS Java on IE6

MS Java in action

And there we go, Microsoft Java is working.

Thanks to the java test site that I’ve used over the years:

http://hp.vector.co.jp/authors/VA012735/applet/applets_en.htm

Hosting Minecraft as an experiment

So in the latest gamer news, everyone is freaking out about Valve allowing mod developers charge.  It’s amazing how quickly it’s fragmented the community in what was at 2 days before a Valve/GabeN worshiping reddit. (here/here/here/here and a rebuttal)

In the middle of all of this I saw this comment in passing:

Remember how that made me leave Mojang?

Remember how that made me leave Mojang?

So yeah, I never followed the whole Minecraft community thing, but apparently people were hosting servers, then asking users to pay for using mods, and even for using basic items.  And since most people who love Minecraft out there are kids, they were paying with their parents credit cards all over the place for server time, and server mods and whatnot, the parents would find out, and them blame Mojang over the entire thing.  So they banned paying servers (at least from what I understood).

So out of curiosity, since I’ve only really played single player, I thought I’d see how hard it is to run a Minecraft server.

First I’m going to create a Debian 7 VM on my ESXi server.  Nothing too fancy, I have an 8 core box with 8GB of ram, so I was thinking 2 vCPU’s and 384MB of ram, and a 4GB disk.  I mean it’s a simple game, how much can it need, right?

Turns out, it wants a LOT more.

So the install of the OS went pretty smooth, then I have to install Java, which is pretty simple:

apt-get install default-jdk

With that done, the next thing to do is download the server jar file from the download site, or for the purpose of my test, I’m using version 1.8.4.

When I went to run it however, I saw the recommended flags:

java -Xmx1024M -Xms1024M -jar minecraft_server.jar nogui

Ouch.  Yes this thing does expect 1GB of ram.  Ok, so I have to RAM and CPU to spare, so I went ahead and gave it 2GB (since I installed the x86 version of Debian..) and 4 vCPUs.

The next thing for me to do was to set it up on the internet, since I’m not in the office.  I have a VM out on the internet, with an OpenVPN back to my ESXi box for my email.  So without trashing my nat I could get xinetd do the dirty work with this simple entry:

 

root@VPS:/etc/xinetd.d# cat minecraft 
service minecraft
{
    disable         = no
    type            = UNLISTED
    socket_type     = stream
    protocol        = tcp
    user            = nobody
    wait            = no
    redirect        = 192.168.1.139 25565
    port            = 25565
}

Then restart xinetd like this:
 /etc/init.d/xinetd restart

Now with Minecraft running on my ESXi server, and my VPS now configured to forward traffic to the ESXi box over the OpenVPN connection I was all set to go!

And I was able to connect, and all was ‘good’.  But then checking the server…

htop on my Minecraft server

htop on my Minecraft server

545Mb of RAM!  And this is with one user!  And look @ the CPU.  Wow no kidding!

And then I noticed something else, the email performance went from OK to horrible.  I spent a lot of time playing with MTU’s receive and send buffers, and other ‘magic’ trying to get something working.  Since my ESXi server doesn’t have a direct internet connection (yuck) I’m in a shared office so it’s not only behind NAT, but I have a DLINK that I use behind their NAT.  And while the UDP protocol ‘works’, changing it to TCP gave me a 5x speed increase.

Very unexpected.

My own world..

My own world..

And not to forget, some helpful stuff for the server:

How do you shut down safely, from the console?

stop

What is the best way to run the server?

Probably behind screen. I started it from /etc/rc.local like this:

/usr/bin/screen -dmS minecraft /usr/local/minecraft/start.sh

start.sh is simply:

#!/bin/sh
cd /usr/local/minecraft/
java -Xmx1024M -Xms1024M -jar minecraft_server.jar nogui

How do I connect to the console?
screen -r minecraft

Remember in this case we gave the screen session a name so it’s easy to find.

How do I disconnect from the console

CONTROL+A+D

Why am I doing this?

I have no idea why. Honestly I find crafting in a game kind of tedious, but setting up a VPN, server and whatnot is more fun to me.

How about network performance?  Since it’s just me, I thought I should look inside the tunnel for a minute and see how big the capture file is:

# tcpdump -s 1520 -w 1.cap -n -i tun0 port 25565 &amp; sleep 60;kill %1

This will run tcpdump for a minute on the default minecraft port, then after 60 seconds end the capture.

# ls -alh *.cap
-rw-r--r-- 1 root root 1.6M Apr 26 16:00 1.cap

Wow that was bigger than I thought. No wonder Minecraft people are always crying about latency! That translates to 213,33 Kbps or 0.21 Mbps.

Can it be compressed?

# gzip 1.cap
# ls -alh *.cap.gz
-rw-r--r-- 1 root root 680K Apr 26 16:00 1.cap.gz

Which then translates into 91,11 Kbps or 0.09 Mbps. Why people don’t compress their network stuff is beyond me, but then again what do I know?

I guess the next step would be to combine this with stunnel, which not only can encrypt the traffic, but compress it as well.

Microsoft Java

J++

I’ve been meaning to write something about the whole Microsoft foray into the Java language, and where we are today as a result. I know in 2012 it is hard to imagine a world where Java wasn’t just ignored or marginalized (don’t even pretend that Oracle buying out SUN didn’t drive people away) but rather there was a lot of excitement built around Java, and all the language companies were getting behind Java.  Yes this included Borland, and of course Microsoft.  And not to be left out of making a compiler, but Microsoft also wrote their own JVM, or Java virtual machine runtime.  Even more un Microsoft like, is that they provided Internet Explorer and their java on the Macintosh!

“Microsoft is offering a real Mac program at an incredibly attractive price: free. Plus, it comes with a few nifty tools you won’t find at http://www.netscape.com/: A good Java virtual machine and just-in-time compiler, which allows you to run Java applets anywhere, not just in your browser, and even a small but robust Web server.”MacWEEK “Microsoft may have won the browser war”

Joanna Pearlstein February 28, 1997

In this brief period of cross platforming products, Microsoft also had Internet explorer for the SUN, and HP workstations (Solaris/HPUX).Even back in 1997 – 1998 Microsoft was going strong on Java, to the point they were re-writing their website to use as much java/javascript as possible, they had even created DHTML the underpinnings for what we call AJAX today.  Microsoft however didn’t capitalize a new server platform and branding for their J++, but rather relied on NT 4.0 and IIS.  Not that this was ‘bad’ or unexpected, but notice that they considered this a lesson learned in marketing and pushed for a .net server although the car people had their win to rename it to 2003.The other thing of course, was that from the 1.0 to 1.1 specification of Java, Microsoft didn’t wait for SUN to fix the language, they instead took it upon themselves to do so, and of course added in lots of Windows only functionality. This of course led to the protracted lawsuit that won Sun a nice chunk of cash, and pushed Microsoft out of the Java business. Not to have had all that effort wasted, Microsoft then took the JVM and retooled it into the .net platform that we all know and love today.  And while SUN was busy trying to make Java the new business COBOL, Microsoft has been making the .net platform business cobol, by letting people choose what language they wish to use, even… COBOL (from Fujitsu IIRC).Installing J++

So on the weekend I came across this book on “Web programming” which I’d usually laugh at for being obsolete and ignore, but it proudly mentions that the included CD includes the ‘publishers’ version of J++ 1.0!  So I checked the book, and yes the CD is still there! So for the 2-3 people that care, I even packaged this fifteen year old oddity. It’s demanding requirements are NT 4.0, or Windows 95 with sixteen (Yes sixteen!) megabytes of ram, and 100MB of free disk space.  Personally I just installed it in a blank NT 4.0 unpatched VM. It’ll install IE 3 along with it, and version 1.00.6211 of the Microsoft JVM.

Naturally even emulated, on a 3Ghz CPU with a gigabytes worth of RAM it runs and compiles quite quickly.

Hello!

I know it isn’t much  to look at, and the download is small it is quite neat for the age/size… But yes, even ‘modern’ java can run “well behaved” J++ apps..

Running my J++ 1.0 hello program

As part of the trial though Microsoft had to pull everything with the JVM in it, and that included IE 3.0 “full”, 4.0 and 5.0/5.5 .. Even Windows 2000 sp3, and prior had to go.  Yes this is also why Office 97 & SQL 7 are gone from the MSDN downloads.  Maybe its my nostalgia but I really did like the 1997-1998 era and their applications.  The only reason I “upgraded” out of Outlook 98 was that 2003 can connect to Exchange 2003/2007 servers with a built in HTTP connection so I don’t have to VPN to send/receive email.

I nearly forgot about this ancient page with some applettes. Amazingly they still run.. Since my experience with Java has been largely write once, debug everywhere…

Update for jdosbox 0.74.24

Awesome news that somehow slipped by, jDOSBox has been updated!  Check out these awesome features!

0.74.24 July 2, 2011 
* Fixed some mouse sensitivity issues 
* Reduced flicker 
* Ported Tandy sound 
* Ported IPX

I’ve updated my site, and I’ll have to see what it takes to get the IPX going…!  For those of you who want to run it on your own, download it here.

Update for jdosbox 0.74.23

Woo new update, and here is what’s changing!

0.74.23 June 5, 2011
* Fixed a serious memory error that mostly affected EGA era games
* Increased performance of the dynamic core a bit
* Improved audio by reducing stuttering on some games

0.74.22 May 29, 2011
* More fixes for Windows 3.11

0.74.21 May 27, 2011
* Fix for Windows 3.11 that affected Freecell

0.74.20 May 27, 2011
* New dynamic core, most games are 2x faster than build #19
* Fixed lots of minor bugs, mainly around installers

Download it here!

So i did update my jdosbox stuff, for anyone that enjoys this stuff. Hopefully it won’t blink like crazy on OS X.

 

Synchronet in a bubble

Or at least that’s what I’m going to call it.

Click to run!

Basically it’s a super minimally configured copy of Synchronet configured to run on jdosbox.  There is no external ‘world’ to talk to.  There isn’t even any ‘persistence’ all changes will be lost after a simple refresh.

There also isn’t any user id’s it’s completely virgin!

Simply click on the picture, and hit the space bar once the menu comes up and you can login!  Since it’s just console access there is no modem, dialing, tcpip stuff to worry about.  If you can run java you can run this.

So why would I or you bother?  I guess its an importunity to suffer through the BBS experience.  Maybe I’m just leading up to something else, something really involved to setup so for most people this click & go java thing is about as close as they want to go to ‘feel’ it and it doesn’t involve any actual setup, nor any programs to configure.  And of course, you cannot ‘trash’ this thing beyond hope as the moment you refresh your browser, it’ll reload the image and you are back to the start.

And maybe it’s a sign of the times as well.  Comparing a BBS install to any *AMP install of today really shows in some way just how far we’ve come, in terms of installation.  Not to mention back in the BBS days nobody used SQL, everyone wound up writing their own multiuser database.  Actually in retrospect I have to wonder why more were not based around Dbase/Clipper/FoxPro.

At any rate, it’s really impressive to see that Synchronet is not only still going but thriving out there.  And the center of it all is DOVENET.  A quick and easy way of distributing messages easily can make a community…   Which sadly this thing doesn’t have any network IO so … no community.  See it’s like a bubble, easy to pop, and in it’s own little brief world.

For anyone crazy enough to take the plunge you could download version 2 of Synchronet right here. Then you can telnet to vert.synchro.net and setup an ID for your prospective BBS.  The best part is the simplicity of getting messages in & out… FTP!  You can even use ncftp to make scripts to send / get messages.  I cooked up something terribly simple to do that here.

While I did add TradeWars 2002 as a door, but you’ve also got the other defaults Synchronet Blackjack!, Domain Poker,Supernet Centipede..

For all you TradeWars 2002 fans….

Click to play!

 

I just found out that jdosbox can do the ‘boot’ command.  So as a test here is TradeWars 2002 running in single user mode.  I suppose I could go all out and make a standalone single node BBS with tradewars configured for the ‘full effect’.

But for now this will have to do.

Click the picture to launch! (needs Java).

Retro: a PC/XT emulator

A friend of mine just pointed me to Retro, a PC/XT emulator written in Java.

It’s got a few games ready to roll, which is cool. While not a full featured as dosbox, it does feel significantly smaller, compared to jdosbo’s 2mb image. And since it’s browser based, it’s not like it’s all that difficult to check out.