MMURTL V1.0
Chapter 1
Richard A. Burgess
Previously published by MacMillan Publishing/SAMS Computer Books as
"Developing Your Own 32 Bit Computer Operating System"
Author: Richard A. Burgess - Copyright ©1995 Richard A. Burgess,
Copyright © 1999, 2000, 2001, IP Data Corp. & Richard
A. Burgess
All Rights Reserved
Portions of this electronic book were formatted with software
produced by:
Adobe Corporation for the Acrobat Reader Version,
and IP Data Corp. & Microsoft Corp. for
the Word and HTML versions.
Copyright 1999, IP Data Corp.
No portion of this book or software may be placed on a public retreival
system
without the express written permission of IP Data Corp.
Chapter 1, Introduction
Overview
Computer programmers and software engineers work
with computer operating systems every day. They use them, they work with
them, and they even work "around" them to get their jobs done. If you're
an experienced programmer, I'm willing to bet you've pondered changes you
would make to operating systems you use, or even thought about what you
would build and design into one if you were to write your own.
You don't have to be Albert Einstein to figure out
that many labor years go into writing a computer operating system. You
also don't have to be Einstein to write one, although I'm sure it would
have increased my productivity a little while writing my own. Unfortunately,
I have Albert's absent-mindedness, not his IQ.
Late in 1989 I undertook the task of writing a computer
operating system. What I discovered is that most of the books about operating
system design are very heavy on general theory and existing designs, but
very light on specifics, code, and advice into the seemingly endless tasks
facing someone that really wants to sit down and create one. I'm not knocking
the authors of these books; I've learned a lot from all that I've read.
It just seemed there should be a more "down to earth" book that documents
what you need to know to get the job done.
Writing my own operating system has turned into
a very personal thing for me. I have been involved in many large software
projects where others conceived the specs, and I was to turn them into
working code and documentation. In the case of my own operating system,
I set the ground rules. I decided what stayed and what went. I determined
the specifications for the final product.
When I started out, I thought the “specs” would
be the easy part. I would just make a list, do some research on each of
the desirables, and begin coding. NOT SO FAST THERE, BUCKO!
I may not have to tell you this if you've done some
serious "code cutting," but the simplest items on the wish list for a piece
of software can add years to the critical path for research and development,
especially if you have a huge programming team of one person.
My first list of desirables looked like a kid's wish
list for Santa Claus. I hadn't been a good-enough boy to get all those
things (and the average life expectancy of a male human being wouldn't
allow it either). Realizing this, I whittled away to find a basic set of
ingredients that constitute the real make-up of the "guts" of operating
systems. The most obvious of these, as viewed by the outside programmer,
are the tasking model, memory model, and the programming interface. As
coding began, certain not-so-obvious ingredients came into play such as
the OS-to-hardware interface, portability issues, size and speed.
One of the single most obvious concerns for commercial
operating system developers is compatibility. If you write something that
no one can use, it's unlikely they'll use it. Each of the versions
of MS-DOS that were produced had to be able to run software written for
previous versions. When OS/2 was first developed, the "DOS-BOX" was a bad
joke. It didn't provide the needed compatibility to run programs that businesses
depended on. In the early days, Unix had a somewhat more luxurious life,
because source-code compatibility was the main objective; you were pretty
much expected to recompile something if you wanted it to run on your particular
flavor of the system. I'm sure you've noticed that this has changed substantially.
I did not intend to let compatibility issues drive
my design decisions. I was not out to compete with the likes of Microsoft,
IBM, or Sun. You, however, may want compatibility if you intend to write
your own operating system. It's up to you. One thing about desiring compatibility
is that the interfaces are all well documented. You don't need to design
your own. But I wanted a small API. That was part of my wish list.
Along my somewhat twisted journey of operating-system
design and implementation, I have documented what it takes to really write
one, and I've included the "unfinished" product for you to use with very
few limitations or restrictions. Even though I refer to it as unfinished,
it's a complete system. It's my belief that when a piece of software is
"finished" it's probably time to replace it due to obsolescence or lack
of maintenance.
What This Book Is About
In this book I discuss the major topics of designing your own OS: the
tasking
model, the memory model, programming interfaces, hardware
interface, portability issues, size and speed. I back
up these discussions showing you what I decided on (for my own system),
and then finally discuss my code in great detail. Your wish list may not
be exactly like mine. In fact, I'm sure it won't. For instance, you'll
notice the ever so popular term "object oriented" blatantly missing from
any of my discussions. This was completely intentional. It clouds far too
many real issues these days. Don't get me wrong - I'm an avid C++ and OOP
user. But that's not what this book is about, so I've removed the mystique.
Once again, your goals may be different.
Throughout this book, I use the operating system
I developed (MMURTL) to help explain some of the topics I discuss. You
can use pieces of it, ideas from it, or all of it if you wish to write
your own.
I have included the complete source code to my 32-bit;
message based, multitasking, real-time, operating system (MMURTL) which
is designed for the Intel 386/486/Pentium processors on the PC Industry
Standard Architecture (ISA) platforms.
The source code, written in 32-bit Intel based assembly
language and C, along with all the tools necessary to build, modify, and
use the operating system are included on the accompanying CD-ROM. The hardware
requirement section below tells you what you need to run MMURTL and use
the tools I have developed. It's not really required if you decide
on a different platform and simply use this book as a reference.
One thing I didn't put on my wish list was documentation.
I always thought that many systems lacked adequate documentation (and most
of it was poorly written). I wanted to know more about how the system worked
internally, which I felt, would help me write better application software
for it. For this reason, I have completely documented every facet of my
computer operating system and included it here. You may or may not want
to do this for your system, but it was a "must" for me. I think you will
find that it will help you if you intend to write your own. Well-commented
code examples are worth ten times their weight in generic theory or simple
text-based pseudo-algorithms.
The “Architecture and General Theory” section of this
book along with the sections that are specific the MMURTL operating system,
will provide the information for you to use my system as is, redesign it
to meet your requirements, or write your own. If you don't like the way
something in my system is designed or implemented, change it to suit your
needs. The only legal restriction placed on your use of the source code
is that you may not sell it or give it away.
Who Should Use This Book
This book is for you if you are a professional programmer
or a serious hobbyist, and you have an interest in any of the following
topics:
-
Writing a 32-bit microcomputer operating system
-
80386/486/Pentium 32 bit assembly language (an assembler is included with
source).
-
The C Programming language (a C compiler is included w/ source)
-
Intel 80386/486/Pentium Paged memory operation and management using the
processor paging hardware
-
Intel 80386/486/Pentium 32-bit hardware and software task management using
the processor hardware task management facilities,
-
Embedded or dedicated systems using the 32 bit PC ISA architecture real-time,
message based operating systems
-
PC Industry Standard Architecture hardware management including: DMA Controllers,
Hardware Timers, Priority Interrupt Controller Units, Serial and Parallel
Ports, Hard/Floppy disk controllers
-
File systems (a DOS File Allocation Table compatible file system in C is
included)
You may note that throughout this book I refer to
the 386, 486, and Pentium processors as if they were all the same. If you
know a fair amount about the Intel 32-bit processors, you know that a quantum
leap was made between the 286 and 386 processors. This was the 16- to 32-bit
jump.
The 486 and Pentium series have maintained compatibility
with the 386-instruction set. Even though they have added speed, a few
more instructions, and some serious "turbo" modifications, they are effectively
super-386 processors. In my opinion, the next quantum leap really hasn't
been made, even though the Pentium provides 64-bit access.
My Basic Design Goals (yours may vary)
My initial desires that went into the design of
my own operating system will help you understand what I faced, and what
my end result was. If you're going to write your own, I recommend you take
care not to make it an "endless" list. You may start with my list
and modify it, or start one from scratch. You can also gather some serious
insight to my wish list in chapter 2, General Discussion and Background.
Here was my wish list:
-
True Multitasking - Not just task switching between programs, or
even sharing the processor between multiple programs loaded in memory,
but real multi-threading - the ability for a single program to create several
threads of execution that could communicate and synchronize with each other
while carrying out individual tasks. I'll discuss some of your options,
and also what I decided on, in chapters 3 (Tasking Model), and 4 (Interprocess
Communications).
-
Real-time operation - The ability to react to outside events in
real time. This design goal demanded MMURTL to be message based. The messaging
system would be the heart of the kernel. Synchronization of the messages
would be the basis for effective CPU utilization (the Tasking Model). Real-time
operation may not even be on your list of requirements, but I think you'll
see the push in industry is towards real-time systems, and for a very good
reason. We, the humans, are outside events. We want the system to react
to us. This is also discussed in chapters 3 (Tasking Model), and 4 (Interprocess
Communications).
-
Client/Server design - The ability to share services with multiple
client applications (on the same machine or across a network). A message-based
operating system seemed the best way to do this. I added the Request and
Respond message types to the more common Send and Wait found on most message
based systems. Message agents could be added to any platform. If you don't
have this requirement, the Request and Respond primitives could be removed,
but I don't recommend it. I discuss this in chapter 4 (Interprocess Communications).
-
Common affordable hardware platform with minimal hardware requirements
- The most common 32-bit platform in the world is the PC ISA platform.
How many do you think are out there? Millions! How much does a 386SX cost
these days? A little more than dirt, or maybe a little less? Of course,
you may have a different platform in mind.The hardware interface may radically
change based on your choice. I discuss this in chapter 6, “The Hardware
Interface.”
-
Flat 32-Bit Virtual Memory Model - Those of you that program on
the Intel processors know all about segmentation and the headaches it can
cause (Not to mention Expanded, Extended, LIM, UMBs, HIMEM, LOMEM, YOURMOTHERSMEM,
and who knows what other memory schemes and definitions are out there).
Computers using the Intel compatible 32-bit processors are cheap, and most
of them are grossly under used. MMURTL would use the memory paging capabilities
of 386/486 processors to provide an EASY 32-bit flat address space for
each application running on the system. Chapter 5 covers memory management,
but not all the options. Many detailed books have been written on schemes
for memory management, and I stuck with a very simple paged model that
made use of the Intel hardware. You could go nuts here and throw in the
kitchen sink if desired.
-
Easy programming - The easiest way to interface to an operating
system from a procedural language, such as C or Pascal, is with a procedural
interface. A procedural interface directly into the operating system (with
no intermediate library) is the easiest there is. Simple descriptive names
of procedures and structures are used throughout the system. I also wanted
to maintain a small, uncluttered Applications Programming Interface (API)
specification, adding only the necessary calls to get the job done. Power
without BLOAT... the right mix of capability and simplicity. I would shoot
for less than 200 basic public functions. I discuss some of your options
in chapter 8, “Programming Interfaces.”
-
Protection from other programs - But not the programmer. I wanted
an OS that would prevent another program from taking the system down, yet
allow us the power to do what we wanted, if we knew how. This could be
a function of the hardware as well as the software, and the techniques
may change based on your platform and processor of choice.
-
Use the CPU Instruction Set as Designed - Many languages and operating
systems ported between processors tend to ignore many of the finer capabilities
of the target processor. I didn't want to do this. I use the stack, calling
conventions, hardware paging, and task management native to the Intel 32-bit
x86 series of processors. I have attempted to isolate the API from the
hardware as much as possible (for future source code portability). You
may want your operating system to run on another platform or even another
processor. If it is another processor, many of the items that I use and
discuss may not apply. I take a generic look at it in chapter 6, “The Hardware
Interface,” and chapter 8, “Programming Interfaces.”
-
Simplicity - Keep the system as simple as possible, yet powerful
enough to get the job done. We also wanted to reduce the "jargon" level
and minimize the number of terse, archaic names and labels so often used
in the computer industry.
-
A Brief Description of MMURTL
From my wish list above, I put together my own operating system. I
will describe it to you here so that you can see what I ended up with based
on my list.
MMURTL (pronounced like the girl's name Myrtle)
is a 32-bit, Message based, Multitasking, Real-Time, operating system designed
around the Intel 80386 and 80486 processors on the PC Industry Standard
Architecture (ISA) platforms. The name is an acronym for Message based
MUltitasking, Real-Time, kerneL. If you don't like my acronym, make up
your own! But, I warn you: Acronyms are in short supply in the computer
industry.
MMURTL is designed to run on most 32-bit ISA PCs
in existence. Yes, this means it will even run on an 80386SX with one megabyte
of RAM (although I recommend 2Mb). Then again, it runs rather well on a
Pentium with 24 MB of RAM too. If you intend to run MMURTL, or use any
of the tools I included, see the “Hardware Requirements” section later
in this chapter.
MMURTL is not designed to be compatible with today's
popular operating systems, nor is it intended to directly replace them.
It is RADICALLY different in that SIMPLICITY was one of my prime design
goals.
If you don't want to start from scratch, or you
have an embedded application for MMURTL, the tools, the code, and the information
you need are all contained here so you can make MMURTL into exactly what
you want. Sections of the source code will, no doubt, be of value in your
other programming projects as well.
Uses for MMURTL
If you are not interested in writing your own operating
system (it CAN be a serious time-sink), and you want to use MMURTL as is
(or with minor modifications), here's what I see MMURTL being used for:
-
MMURTL can be considered a general-purpose operating system and dedicated
vertical applications are an ideal use.
-
It is an ideal learning and/or reference tool for programmers working in
32-bit environments on the Intel 32-bit, x86/Pentium processors, even if
they aren't on ISA platforms or using message based operating systems.
-
MMURTL can be the foundation for a powerful dedicated communications system,
or as a dedicated interface between existing systems. The real-time nature
of MMURTL makes it ideal to handle large numbers of interrupts and communications
tasks very efficiently.
-
Dedicated, complex equipment control is not beyond MMURTL's capabilities.
-
Vertical applications of any kind can use MMURTL where a character based
color or monochrome VGA text interface is suitable.
-
MMURTL would also be suitable for ROM based embedded systems (with a few
minor changes). One of the goals was to keep the basic system under 192Kb
(a maximum of 128K of code, and 64K of data), excluding dynamic allocation
of system structures after loading. The OS could be moved from ROM to RAM,
and the initialization entry point jumped to. If you're not worried about
memory consumption, expand it to your heart's desire.
-
In the educational field, MMURTL can be used to teach multitasking theory.
The heavily commented source code and in-depth theory covered in this book
makes it ideal for a learning/teaching tool, or for general reference.
-
And of course, MMURTL can be the basis or a good starting point for you
very own microcomputer operating system.
Similarities to Other Operating Systems
MMURTL is not really like any other OS that I can
think of. It's closest relative is CTOS (Unisys), but only a distant cousin,
and only because of the similar kernel messaging schemes. Your creation
will also take on a personality of it's own, I'm sure.
The flat memory model implementation is a little
like OS/2 2.x (IBM), but still not enough that you could ever confuse the
two.Some of the public and internal calls may resemble UNIX a little, (or
even MS-DOS), but still, not at all the same.
The file system included with MMURTL uses the MS-DOS
disk structures, but only out of convenience. It certainly wasn't because
I liked the design or file name limitations. There are some 100 million+
disks out there with MS-DOS FAT formats. This makes it easy to use MMURTL,
and more importantly, eases the development burden. No reformatting or
partitioning your disks. You simply run the MMURTL OS loader from DOS and
the system boots. Rebooting back to DOS is a few keystrokes away. If you
want to run DOS, that is.
MMURTL has it's own loader to boot from a disk to
eliminate the need for MS-DOS altogether. If you don't want to write a
completely new operating system, some serious fun can had by writing a
much better file system than I have included.
Hardware Requirements
This section describes the hardware required to run MMURTL (as included)
and to work with the code and tools I have provided. If you intend to build
on a platform other that described here, you can ignore this section.
The hardware (computer motherboard and bus design)
must be PC ISA (Industry Standard Architecture), or EISA.Other 16/32 bit
bus designs may also work, but minor changes to specialized interface hardware
may be required.
The processor must be an Intel 80386, 80486, Pentium,
or one of the many clones in existence that executes the full 80386 instruction
set. This includes Cyrix, AMD, IBM and other clone processors.
VGA videotext (monochrome or color) is required.
MMURTL accesses the VGA text memory in color text mode (which is the default
mode set up in the boot ROM if you have a VGA adapter).
One Megabyte of RAM is required, 2 MB (or more)
is recommended. MMURTL will handle up to 64 Megs of RAM. MMURTL itself
uses about 300K after complete initialization.
The Floppy disk controller must be compatible with
the original IBM AT controller specification (most are). Both 5.25" and
3.5" are supported. The hard disk controller must be MFM or IDE (Integrated
Drive Electronics). IDE is preferred. Some RLL controllers will not work
properly with the current hard disk device driver.
A standard 101 key AT keyboard and controller (or
equivalent) is required.
The A20 Address Gate line must be controllable as
documented in the IBM-PC AT hardware manual, via the keyboard serial controller
(most are, but a few are not). If yours isn't, you can change the code
as needed, based on your system manufacturer's documentation (if you can
get your hands on it).
8250, 16450 or 16550 serial controllers are required
for the serial ports (if used).
The parallel port should follow the standard IBM AT
parallel port specification. Most I have found exceed this specification
and are fully compatible.
MMURTL does NOT use the BIOS code on these machines.
Full 32-bit device drivers control all hardware. Hence, BIOS types and
versions don’t matter.
End of Chapter 1