My wonderful Meg kitty.

(no subject)

I plan to have a very primative 8086 capable bootloader done by the end of this weekend. It occurred to me that I ought to let people know how I'm designing bootloaders.

The bootloader should have a limited purpose in life -- I plan for it to provide an address for additional parts of the operating system (or program) to load into. Following the standard of int 13, it seems prudent for that address to be located in ES:BX and that the address located here ought to be as low in memory as possible.

In the remaining 512 bytes should be the portion to load from disk into memory. This can be as primative or as fanciful as one wishes. My initial format is to simply load all possible contiguous sectors from 1 into memory. After having loaded that portion, there should be a jump to ES:BX.

Using this initial boot code and an appropriate bootloader for the disk at hand, one should be able to load anything they'd like into memory -- a different operating system, an embedded program, so on and so forth.

From here, Ultimate will formally map memory, load -- but not initialize -- device drivers and an interface. I intend to maintain a switch to a hardware shell in here. What I mean is some way to control the initialization of devices and pass commands directly to the hardware if one were inclined. It's primarily intended for debugging questionable hardware. One thing I always liked about MS-DOS was that it did the very minimum that an operating system needed to do. When I had problems with hardware before I knew linux, I'd boot a DOS disk and play around with things.

Once that's done, we'll get into implementing the lofty goals I started off with.

I've been thinking that I should really package RAWRITE.EXE along with it for writing that floppy image, but I'm under the impression that everyone reading this has access to a linux box. How true is that?

Also, I made some minor changes to the Ultimate-0.0.1 tarball. Turns out I've been using a really lenient BIOS that let me get away with a call instead of a jump as the first instruction and without the signature at the end of the 512b block.
My wonderful Meg kitty.

(no subject)

Also, I'd appreciate reports on:
1) If you can get it to boot (one person hasn't, but it seems like the x86 emulator he's using is the problem),
2) If you can't, what the stats of the box are,
3) If you can, what it outputs,

The output is a memory map. I realized I screwed up somewhat and my function to print out the addresses is actually printing them backwards. Heh -- whoops... ; )
My wonderful Meg kitty.

Ultimate 0.0.1!

In 240 bytes, it resolves where in memory it's been loaded, queries the BIOS for the memory map, determines free addresses, and (soon) will construct a bitmap and use int 13 to load the remainder of the floppy into the earliest possible segment.

This uses the int 15, E820 extension of fairly modern bioses which requires a 32-bit processor and a BIOS that supports that function. It works on an old Pentium-120 I have and bochs, so any x86 later than 1995 might be able to boot this.

A minor tweak in the code using a different interrupt and the elimination of a couple of 32-bit instructions will make this code usable on any x86 with a BIOS.

So, without further ado, I give you Ultimate 0.0.1 with source and a usable floppy image (which is really just nasm output assembled to a 1.44mb image of zeros).

Here's my roadmap:

1) write a more generic bootloader using int 10h, AH=C7h
2) finish C code to take information from bootloader and generate a memory map
3) use int 13h to load the driver interface code I've written, load subsequent sectors into memory, and jump to the beginning of the loaded code.
My wonderful Meg kitty.

(no subject)

So, many of you may be thinking that it's April 1st and Ultimate still isn't up...

And you'd be right! 8)

The bootloader is done and part of the device interface needs to be written. I expect it before not too much longer.

Those who want to know why it's late may check my journal. The post after it will be much more informational.
My wonderful Meg kitty.


Having finished a couple of potential engines to drive adaptive behaviors, I've started work on a simulator. Why a simulator? Why not use bochs? Why not write a few drivers and put it on a spare machine?

Well, honestly, I'm still not sure how I want my driver model to work. Here's what I've been considering:

I've been considering a system not unlike older architectures where access to drivers is memory mapped. For example, if I want to write to disk (which we'll whimsically call device 3), then I'll place in address 3 a position independent pointer to the data I want to copy and a size reference. Instead of the usual trap system, the memory access causes a fault. The buffer pointed to is copied into kernel space and the hardware access continues as planned without requiring further interaction from the program. If I want to know the return value, I'll read from address 3. If the driver hasn't finished, it'll block.

I really like the idea of writing things to memory as opposed to a trap system. It seems like an easier way to handle hardware access.

I suspect a more flexible way to write this would be to have a page which is broken up into structures of three integers and a pointer. The first integer encodes the desired device. The second integer encodes the desired function of the device (read, write, &c). The third integer is a count and the pointer points to a buffer within the process memory space. This also allows the device to return more information.

There is a downside to this approach: Typically the reason NULL and pointers near 0 cause segmentation faults is because the zero page is an inaccessible page by design. In deference to this strategy which nicely traps NULL pointers, I think a different location would be needed. Perhaps an address provided by the operating system? I haven't quite decided yet.
My wonderful Meg kitty.

(no subject)

I threw away a lot of code today.

I've had a somewhat operational system for a little bit now that's single process and single user, but with the ability to be expanded.

The reason I threw away all that code was because I came to the realization that my solution to expandability wasn't expansive at all. I had been using function pointers for a couple of different tasks and had an overly elaborate way of registering services with the operating system. It had bugged me for a while and it occurred to me that I'm not creating anything new with it. Instead, I'm hacking together some third-rate implementation of object-oriented concepts with all the inherent problems therein.

One thing that I've seriously considered for Ultimate is self modifying OS code. This will, no doubt, sound absolutely insane to anyone in their right mind. I'm not entirely in disagreement, but I believe that in order for an OS to be of any significant worth in this day and age, it must be capable of actively changing the way it reacts to certain problems.

It's not enough to provide a series of modules and load them in as the time comes. Computers are fast enough and machine learning algorithms good enough that there's no reason that the majority of things such as page swapping algorithms shouldn't be dynamically adjustable based on data the system gathers about how it runs. There are situations that the OS will get itself into that cannot necessarily be planned for (cf. the THE operating system), but it should be able to get itself out.

I would go so far as to say that hard coding things like page replacement algorithms as opposed to page replacement managers is to remove any hope of actually creating something new and interesting in favor of something usable.

This may sound silly, but I don't think Ultimate is intended to necessarily be something usable. It's intended to be a platform on which we can put forth all the crazy ideas we have about how OSes ought to work in complete willful defiant ignorance of the sorts of things that 60+ years of OS papers have said that an OS Should Have.

If it turns out to be usable, so much the better.

In large part, I think this is the misunderstanding that have led some people to post things like "Use NetBSD" or "Use Linux".

Completely out of curiousity: Does anyone else out there share my brain space or am I in danger of losing anyone else's interest in this project? I wonder when not too many people comment on the entries I post ; )
My wonderful Meg kitty.

Virtual machines, application execution, and how it all fits in.

I was asked by wstockall to clarify the differences between my planned system and Xen's hypervisor.

Xen (for those who aren't aware) runs on x86 and creates a virtual machine. It's not at all unlike VMWare in this sense.

It has a hypervisor which (to my understanding) is a layer between the host operating system and the guest operating system. The hypervisor essentially translates the guest OSes traps into executing code on the host OS.

I have proposed what amounts to two parallel layers of abstraction.

The first is that each user is effectively in their own sandbox. My vision for this is roughly equivalent to booting DOS or CP/M on a system. They are effectively the root user in that compartment. We'll call this the user compartment or UC.Collapse )

The second layer is roughly equivalent to the loader. When the user executes a file, this layer loads the file into memory and loads whatever programs are needed to execute it. This is what I call the execution environment loader or EEL.Collapse )

Hopefully that clarifies the distinction between this part of Ultimate and the way that Xen goes about things.
My wonderful Meg kitty.

(no subject)

So here's a fun idea:

AMD's Hammer provides an interesting function. You can load in and save out CPU branch prediction tables.

It occurred to me that it'd be interesting to set up paging and resource allocation in a similar preemptive fashion. Instead of running on a static algorithm, anticipate the processes needs, and allocate beforehand.

As Darrell Long (a professor in my past) has said, the recent past is a good indicator of the near future.

A lot of this would be fairly easy to write (given the plan to virtualize the execution environment), but I imagine that storing the data for easy lookup and minimal resources is a bit more difficult.