<!--
foo
Note: I have changed the style of my comments. The standard color for comments, red, is not readable for large comments.
-->
<style type="text/css">span.Mad70 {
	font-size: small;
	color : Navy;
}</style>

This is a debate on _(Microkernel)s.
It begins with the original TUNES Glossary entry on _(Microkernel)
as written by _(Far).
Follows a response by _(KC5TJA), prompted by _(thin).
An answering to this response was added as bracketed comments by _(Mad70).

<h2>Microkernels</h2>
(original TUNES Glossary entry by _(Far))

Microkernel (also abbreviated K or uK) is the *(term) describing an approach to _(Operating System) design by which the functionality of the system is moved out of the traditional "_(kernel)", into a set of "_(server)s" that communicate through a "minimal" _(kernel), leaving as little as possible in "system space" and as much as possible in "_(user) space".

<h3>Rationale</h3>

Microkernels were invented as a reaction to traditional "monolithic" _(kernel) design, whereby all system functionality was put in a one static program running in a special "system" mode of the processor. The rationale was that it would bring modularity in the system architecture, which would entail a cleaner system, easier to debug or dynamically modify, customizable to _(user)s' needs, and more performant.

<h3>Examples</h3>

Perhaps the best known example of microkernel design is _(Mach), originally developed at CMU, and used in some free and some proprietary _(BSD) _(Unix) derivatives, as well as in the heart of _(GNU) _(HURD). Rumor had it _("<tt>MICROS~1</tt> Windows NT"|Microsoft Windows) would originally have been a microkernel design (that was grown into the bloated thing it is), but this has been denied by _(NT|Microsoft Windows) architect Dave Cutler. Other well-known microkernels include _(Chorus), _(QNX), _(VSTa), etc. Latest evolutions in microkernel design led to things like "nano-_(kernel)" _(L4), or "exokernel" _(Xok), where the _(kernel) is shrunk ever more towards less functionality and less portability.

<h3>Opinionated History</h3>

At one time in the late 1980's and early 1990's, microkernels were the craze in official academic and industrial _(OS) design, and anyone not submitting to the dogma was regarded as ridiculous (at least it seems to me from reading articles from _(OS) conferences, or the _(Minix) vs _(Linux) flamefest; could people help confirm or infirm this impression of mine?). But microkernels failed to deliver their too many promises in terms of either modularity, cleanliness, ease of debugging, ease of dynamic modification, customizability, or performance. This led some microkernel people to compromise by having "single-_(server)s" that have all the functionality, and pushing them inside _(kernel)-space (allegedly _(NT|Microsoft Windows), hacked _(MkLinux)), yielding a usual monolithic _(kernel) under another name and with a contorted design. Other microkernel people instead took an even more radical view of stripping the _(kernel) from everything but the most basic system-dependent interrupt handling and messaging capabilities, and having the rest of system functionality in libraries of system or _(user) code, which again is not very different from monolithic systems like _(Linux) that have well-delimited architecture-specific parts separated from the main body of portable code. With the rise of _(Linux), and the possibility to benchmark monolithic versus microkernel variants thereof, as well as the possibility to compare _(kernel) development in various open monolithic and microkernel systems. people were forced to acknowledge the practical superiority of "monolithic" design according to all testable criteria. Nowadays, microkernel is still the "official" way to design an _(OS), although you wont be laughed at when you show your monolithic _(kernel) anymore. But as far as we know, no one in the academic world dared raise any theoretical criticism of the very concept of microkernel.

<h3>Argumented Criticism</h3>

As people understood that _(kernel)s only introduce (design-time and run-time) overhead without adding any functionality that couldn't be better achieved without it (for several reasons like efficiency, maintainability, modularity, etc), they tried to reduce _(kernel) sizes as much as they could. The result is called a microkernel, which is pure overhead, with no functionality at all. There has thus been a (now waning) craze in _(Operating System) research and development to boast about using a microkernel. 

I contend that microkernels are a deeply flawed idea: instead of removing the _(overhead), they concentrate and multiply it. The overall space/time cost of the _(OS) is not reduced at all, as the functionality has only been moved away from the _(kernel) into "_(server)s"; only now there is an additional overhead in space as well as in time and as in design, to manage the information flow of system services that now needs to go from _(user) to _(kernel) then _(kernel) to _(server). Because of the low _(abstraction level) of microkernels, lots of _(low-level) bindings must be done for "_(server)s" that provide functionality, so nothing is gained at the _(user)/_(server) interface either.

As a result, microkernel-based systems are slower, bigger, harder to program, and harder to customize than monolithic _(kernel)s. The only valid rationale about them is that they encourage some modularity. However, this modularity microkernels enforce on system programmers is of a very _(low-level) kind, which implies the overhead of (un)marshalling, as well as total lack of consistency or trust between communicating _(server)s. In comparison, "monolithic" systems can achieve arbitrary useful modularity with dynamically-loaded _(kernel) code, allowing automatic enforcement of whatever consistency the system programming languages can express (for instance, strong static typing with _(module) scoping in _(Modula-3)-programmed _(SPIN) and _(Standard ML | ML)-programmed _(Fox), or just weak typing with filtered global symbol matching in _(C|C language)-programmed _(Linux)).

Thinking that microkernels may enhance computational performance can stem but from a typical myopic analysis: indeed, at every place where functionality is implemented, things look locally simpler and more efficient. Now, if you look at the whole picture, and sum the local effects of microkernel design all over the place, it is obvious that the global effect is complexity and bloat in as much as the design was followed, i.e. at every _(server) barrier. For an analogy, take a big heavy beef, chop it into small morsels, wrap those morsels within hygienic plastic bags, and link those bags with strings; whereas each morsel is much smaller than the original beef, the end-result will be heavier than the beef by the weight of the plastic and string, in a ratio inversely proportional to the small size of chops (i.e. the more someone boasts about the local simplicity achieved by his K, the more global complexity he has actually added w.r.t. similar design w/o K). Microkernels only generate artificial barriers between functionalities, and any simplicity in _(server)s is only the intrinsic simplicity of provided functionality, that is independent from the existence of _(low-level) barriers around it. Every part of a K-based design is simpler (than a whole system), of course, because the design has butchered the system into small parts! But if one considers same-functionality overall systems, the only thing K does is introduce stupid _(low-level) barriers between services. The services are still there, and their intrinsic complexity isn't reduced: for every small part of a K-based system, one could find a corresponding, smaller or equal part, in a same-functionality non-K system, namely the one that implements the same functionality without having to marshall data to cross barriers. 

Microkernels start from the (Right) idea of having modular _(high-level) system design, and confuse the issue so as to end with the (Wrong) idea of its naive implementation as a _(low-level) centralized run-time _(module) manager, which constitutes a horrible _(abstraction inversion). So they have system programmers manually emulate an asynchronous parallel _(actor) model with coarse-_(grain)ed _(C | C language)-programmed polling processes, instead of directly using a real fine-_(grain)ed _(actor) language with optimizing compiler (_(Erlang), _(Mozart/Oz | Oz), _(Modula-3), some _(concurrent) variant of _(Lisp) or _(ML) or _(Haskell), etc.). The discrepancy between the model and its naive and awkward implementation induces lots of overhead, that get worked around with lots of stupid compromises, with a two-level programming system: objects are segregated into a finite set of _(server)s and a _(kernel), with completely different programming models for combining objects inside a same space and for combining objects not in a same space. Performance gets so bad that most "basic resources" must be statically special-cased in the "microkernel" anyway, and people group as much functionality as they can in every _(server) to not pay the price of inter-_(server) communication during their interaction. Semantics also becomes very difficult to get right, since _(low-level) interactions make a hell out of debugging the already complex _(concurrent) _(actor) model. In the end, people put the whole of _(OS) services in a monolithic "single-_(server)", which completely defeats the whole purpose of a microkernel! As a result, everything gets both more complicated and slower! Of course, the very same conclusion holds for _(kernel)s in general; by pushing the idea of _(kernel)s to its limits, microkernels only end up proving the whole inadequacy of it.

The only possible justification for a microkernel is not technical. It's _(political): a microkernel is the only way to allow with any robustness the existence of black-box proprietary third-party binary _(module)s that access and provide deep system resources without anyone having to disclose source code. Microkernels are technically the worst possible organization for system code of same functionality, and the fact that the proprietary closed-source development model encourages such horrors accounts for the deep evil behind that model. It has been suggested that a psychological reason behind the abstraction inversion is that, by a misled tradition, the "_(Operating System)" community stubbornly refuses to mess with language issues (they claim "language independence"), and stick to designing system interfaces for bit-level languages; but we can also track this want of language "independence" to the political issue of proprietary software, since it is what induces the eager clustering of computing into hermetic fields where no one can modify or adapt (proprietary) code from other fields forces people into the paranoid "trust no one, never cooperate: even if you want to, you can't" behavior.

Latest developments in microkernels (_(L4), _(Xok)) amount to reducing as much as possible the semantics and overhead of the _(kernel), and putting everything in either _(server)s or system libraries. The logical next step beyond these developments would be to reduce the microkernels to zero, naught, nada, and have everything in "_(module)s" that constitute the system, and are _(high-level) concepts without forcibly any obvious, one-to-one direct correspondence between the _(high-level) compile-time _(module)s limits and _(low-level) run-time code barriers. Depending on the point of view, this leaves us either with "monolithic" systems, or with systems without a privileged _(kernel) at all (such as systems built atop the _(Flux) _(OSKit)). Such is the right way, in our opinion: to provide _(high-level) modular design, but without any _(kernel) at all. _(Kernel)s are but a stubborn straightforward _(low-level) implementation of _(module) management, through a centralized runtime message passing agent. Tunes will provide an optimizing compiler so that local message passing, which is only a _(low-level) model for application of a function, will be completely inlined.


<h2>Response to Criticism (of exokernel in the glossary entry)</h2>

(Response by _(KC5TJA))

<span class="Mad70">[You missed completely the point, this glossary entry is not defending _(kernel)s vs microkernels!!! WE OPPOSE BOTH. We propose <strong>_(No-Kernel) systems</strong>. Read below. -- _(Mad70)]</span>

First, I'd like to address the issue that an exokernel is a microkernel.  IT IS NOT.  Please do not confuse the microkernel and exokernel concepts.
<span class="Mad70">[_(Fare), the author of this glossary entry, was not confused at all. I think he included _(exokernel)s into the concept of microkernels because the good idea (downloading application code in _(kernel) space) in this model is not pervasive, but limited to some special class of programs (application services as you noted).  -- _(Mad70)]</span>

The argument that microkernels contain pure overhead and contain no functionality is not only false, it's unfounded.  The author of the above cites no references to support any of his claims.
<span class="Mad70">[On the contrary, there is a proof of concept _(No-Kernel) _(OS), _(GO!), which proves that he was right. See in particular _("GO! OS :: PERFORMANCE RESULTS"|http://goos.sourceforge.net/performance.php). -- _(Mad70)]</span>

On the other hand, _(AmigaOS)'s exec.library and _(QNX)'s _(kernel) are products that are still available today (the latter example is available on a much wider scale of course), that continue to demonstrate the viability and functionality of the microkernel concept.

However, the author makes a good point in the following paragraph, where he details that code requirements hasn't been reduced, but only better partitioned.  Let it be known, however, that a reduction in code footprint has NEVER been a goal of the microkernel research.  The goal was to make a more modular system, that was easier to maintain and update.

<span class="Mad70">[The main point of the criticism are <strong>PROTECTION BARRIERS</strong> (called rings in the Intel x86 architecture): the cost of crossing protection barriers is high, because _(Protection) is a stupid form of _(Security). We propose to eliminate protection barriers. See the above mentioned _(GO!), the _(Lisp)-based _(OS) _(Genera), _(Multipop), the Safe Language Kernel Project _(SLK), Kernel Mode Linux _(KML), _(Language-Based Security), _(Information-Flow Security), _(Capability) and other entries.

Also you <strong>completely</strong> ignore his argument about modularity and concurrency:
<blockquote>
[..] So they have system programmers manually emulate an asynchronous parallel _(actor) model with coarse-_(grain)ed _(C | C language)-programmed polling processes, instead of directly using a real fine-_(grain)ed _(actor) language with optimizing compiler (_(Erlang), _(Mozart/Oz | Oz), _(Modula-3), some _(concurrent) variant of _(Lisp) or _(ML) or _(Haskell), etc.). The discrepancy between the model and its naive and awkward implementation induces lots of overhead, that get worked around with lots of stupid compromises, with a two-level programming system: objects are segregated into a finite set of _(server)s and a _(kernel), with completely different programming models for combining objects inside a same space and for combining objects not in a same space. Performance gets so bad that most "basic resources" must be statically special-cased in the "microkernel" anyway, and people group as much functionality as they can in every _(server) to not pay the price of inter-_(server) communication during their interaction. Semantics also becomes very difficult to get right, since _(low-level) interactions make a hell out of debugging the already complex _(concurrent) _(actor) model. In the end, people put the whole of _(OS) services in a monolithic "single-_(server)", which completely defeats the whole purpose of a microkernel! As a result, everything gets both more complicated and slower! Of course, the very same conclusion holds for _(kernel)s in general; by pushing the idea of _(kernel)s to its limits, microkernels only end up proving the whole inadequacy of it. [..]
</blockquote>
-- _(Mad70)]</span>

The issue of space/time efficiency is of course critical, and is perhaps the <strong>only</strong> reason microkernels have not become more popular.  _(AmigaOS) gets around this by message passing using only pointer swaps (blocks of memory are NOT copied in the system),
<span class="Mad70">[The original Amiga hardware was based on a Motorola 68000 CPU which <strong>misses</strong> a _(MMU) and then memory protection. Thus strictly speaking _(AmigaOS) is <strong>not</strong> a _(kernel) <strong>nor</strong> a microkernel. -- _(Mad70)]</span>
while _(QNX) gets around this by blocking the sender, allowing pages to be remapped between processes safely.  Other microkernel environments, however, COPY memory all over hell and creation, just like the Unix-based operating systems they try so hard to emulate (e.g., early versions of _(Mach), _(VSTa), et. al.).  Such systems are widely known in the microkernel communities as horrifically bad examples of microkernels.  Yet, these are often the most popular amongst the public, and therefore get the most attention, and negative press.

The above author goes on to complain about microkernel-based systems being 
"slower, bigger, harder to program, and harder to customize than monolithic _(kernel)s."  Let's respond to each of these in succession:

<ol>

<!-- 1 --><li>Microkernels are slower invariably when they copy memory all over the place, as discussed above.  When they don't, they're quite often <strong>faster</strong>.  Traditional monolithic kernels have to copy memory all over the place in order to get any kind of interprocess communications done, even sockets, currently the single most popular and most-often used _(IPC) mechanism in _(Unix).

<span class="Mad70">[Try this search on Google and you will surprised: _("\"zero-copy\""|http://www.google.com/search?hl=en&ie=UTF-8&oe=utf-8&q=%22zero-copy%22) (you will see for example: _("Zero copy sockets and NFS patches for FreeBSD"|http://people.freebsd.org/~ken/zero_copy/), _("Zero-Copy TCP in Solaris"|http://citeseer.nj.nec.com/chu96zerocopy.html), _("Linux Kernel: zero-copy TCP"|http://lists.insecure.org/linux-kernel/2000/Sep/0237.html)). -- _(Mad70)]</span>


_(AmigaOS) demonstrates the sheer speed of a microkernel-based environment: my 7MHz Amiga 500 computer, for example, though noticably slower in processing power, still has overwhelmingly superior _(user) response times than my 800MHz AMD Athlon box running Linux 2.4.18 _(kernel), even when running as many as 15 compute-intensive tasks.
<span class="Mad70">[I repeat myself: your Amiga 500 doesn't have an _(MMU) and then it doesn't have protection barriers. -- _(Mad70)]</span>
</li>

<!-- 2 --><li>Microkernel environments are bigger.  There is no evidence of this being the case.  My _(QNX) RtP 6.2 installation consumes a mere 40MB of my harddrive, including the _(GUI).  My Linux installation consumes over 250MB for the same level of service.  And if you think _(QNX) is small, my _(AmigaOS) installation is only 6MB, including the 512K ROM space which must necessarily be included for completeness.

<span class="Mad70">[You are not drawing a clear distinction between _(OS) _(kernel) and applications: this just shows that the usual definition of _(Operating System), when you consider level of service, is at least <em>vague</em> if not <em>meaningless</em>. Have a look at _(Fare) definition of _(Operating System) (common background). Also we are not defending _(Linux)/_(Unix) so this is irrelevant. -- _(Mad70)]</span>
</li>

<!-- 3 --><li>Harder to program.  Every time you write a program that runs in a _(GUI), you're writing a microkernel "server" program.

<span class="Mad70">[Client/server is not the unique model of _(concurrent) programming and it is one of the more difficult.

Citing from _("The Role of Language Paradigms in Teaching Programming (.pdf)"|http://www.info.ucl.ac.be/people/PVR/sigcse2003panel.pdf), pp. 1/2:
<blockquote>
[..]
<h4>1 Joe Armstrong</h4>

Programs that model or interact with the real world need to reflect the _(concurrency) patterns that are observed in the real world. The real world is concurrent - and writing programs to interact with the real world should be a simply a matter of identifying the _(concurrency) in the problem, identifying the message channels and mapping these 1:1 onto the code - the program then almost writes itself.

Unfortunately, concurrent programming has acquired a reputation of being "difficult" and something to be avoided if possible. I believe this is a side-effect of the problems of thread programming in conventional operating systems using languages like _(Java), _(C|C Language), or _(C++). In a concurrent language like _(Erlang), concurrent programming becomes "easy" and becomes the natural way of solving a large class of problems.

Most conventional languages that have primitives for concurrent programming provide only a thin layer to whatever mechanisms are offered by the host operating system. Thus _(Java) uses the _(concurrency) mechanisms provided by the underlying operating system and the inefficiency of _(concurrency) in _(Java) is merely a reflection of the fact that the _(concurrency) mechanisms in the operating system are inefficient.

I believe that _(concurrency) should be a property of the programming language and not something inherited from the _(OS). _(Erlang) is such a language. _(Erlang) processes are extremely lightweight: creating a parallel process in _(Erlang) is about 100 times faster than in _(Java) or _(C++). That's because _(concurrency) is designed into the language and has nothing to do with the host OS. Once you put _(concurrency) into the language a lot of things look very different - concurrent programing becomes easy. This is especially important in programming high-availability real-time or distributed applications where _(concurrency) is inescapable.

[..]
</blockquote>

-- _(Mad70)]</span>

The server consists of an event loop, just like any normal _(GUI) program does.

<span class="Mad70">[This is only what <strong>you</strong> know: there are other models for programming _(GUI)s which escapes the event-loop. See for example the paper: _("Escaping the event loop: an alternative control structure for multi-threaded GUIs (.ps.gz)"|http://www.cs.nyu.edu/phd_students/fuchs/gui.ps.gz) by Matthew Fuchs. Summarizing, on his _("home page"|http://www.cs.nyu.edu/phd_students/fuchs/), about contributions of his PhD dissertation, he writes:

<blockquote>
[..]

How to escape the ubiquitous _(GUI) event loop and eliminate the tortured, dismembered programming style it engenders. The essential realization is that "reactive programming" with callbacks is really a twisted form of _(Continuation-Passing Style), a source code transformation commonly used in compilers for _(functional) languages.

[..]
</blockquote>

See also: the paper <em>Separating Application Code from Toolkits: Eliminating the Spaghetti of Call-Backs</em> about _(Garnet), which use _(programming by demonstration) and _(constraints); _(On Automatic Interface Generation).

-- _(Mad70)]</span> 

They even take the same form. In the case of a microkernel, they even use the same communications channels API, which significantly <strong>simplifies</strong> the program design and implementation.  I can only dream of a day when I can <strong>arbitrarily</strong> intermix GTK, _(CORBA), and arbitrary socket API calls in my software under _(Linux). <span class="Mad70">[See above discussion on _(concurrent) and other styles of programming. And, again, we are not defending _(Linux)/_(Unix) and _(C|C language) _(paradigm) of programming, so this is irrelevant. -- _(Mad70)]</span> As it is, it's patently impossible to do without dedicated, special-purpose libraries like libgnorba.
</li>

<!-- 4 --><li>Harder to customize.  This is a moot issue, as the microkernel has zero say about how to customize the environment.  If the software running on top of the microkernel is designed to emulate a Unix environment, it'll, well, emulate a _(Unix) environment.  This means, editing files in the /etc directory, manipulating symlinks, et. al.  What does make things easier is the ability to run and stop "services" or daemons which implement certain functions.  For example, in _(QNX), you can run and stop _(filesystems|File System) <span class="Mad70">[Filesystems? We propose _(orthogonal persistence) instead. -- _(Mad70)]</span> at any time, change the parallel port behavior by stopping one driver and starting another, etc.  Compared to Linux's _(module) system, it all behaves exactly the same way.  Compared to _(NT|Microsoft Windows)'s "services" system, it all behaves exactly the same way.  The author above is showing a fundamental lack of understanding of how microkernel architecture works. <span class="Mad70">[No, you are showing us a fundamental lack of knowledge and/or understanding of different _(paradigm)s of programming other than that of _(C|C language).
<br>-- _(Mad70)]</span>
</li>

</ol>

The author's use of analogies above are wholly inadequate, and fail to address the core issues surrounding microkernels.  I'm still trying to figure out how beef has anything at all to do with microkernels versus monolithic kernels, in either the space or time domains.

The above essay is so fundamentally flawed in his arguments that I find I can't go on thinking about it currently.  Despite being heavily biased towards the monolithic architecture, the author's intentions were clearly good.  I will openly and honestly admit that microkernels DO have some space/time trade-offs, and that the _(user)-_(kernel)-_(user) transitions will consume additional overhead. <span class="Mad70">[Ah! Better late that never. -- _(Mad70)]</span> But to make a wide generalization that ALL microkernels are big, slow, hard to program, or hard to customize is just so wrong that it hardly warrents explicit justification.
<span class="Mad70">[No, we are not heavily biased towards the monolithic architecture. Please, read sufficiently this _(Wiki|index) and the _("main site"|http://tunes.org) before jumping to unfounded conclusions.
<br>-- _(Mad70)]</span>

<span class="comment">[_(Mad70), I saw that there were some comments about Exokernels that I thought were misleading in the glossary entry, and I asked _(KC5TJA), who is much more knowledgeable about microkernels and exokernels than I am, to make the above entry to discuss this.  He did not know about TUNES's aims and thus was not speaking in that context. I think it is distasteful for you to attack him on that context, but the fault is perhaps mine as I did not pay attention to the "policy" of TUNES. At any rate, I assumed that the cliki was going to carry the same philosophy as the rest of the TUNES website - reviews of languages, OSes, methodologies, and to show both sides of the argument.
<br>p.s.: you can remove my entry if you want.
<br>-- _(thin)]</span>

<span class="comment">[I would like to take this time to thank _(Mad70) for responding. His input gives me something to think about with respect to no-kernels and other related organizational structures. I can see his points, though I do not agree with all of them, especially where he uses the Amiga itself as supporting evidence for his views on no-kernels. I will take some time to consider them.

I have to admit that I have considered no-kernel-like operating environments in the past, and to this date, feel that they do represent the epitome of operating system design philosophy.  However, that merely obviates the need for protection domains.  It does not address the issue of shared resource management, which exokernels specialize in.  It is fully possible to have a no-kernel exokernel design, just as it's possible to have a microkernel running under an exokernel (or vice versa); this is the source of my contention with respect to exokernels.  The quality of being an exokernel is orthogonal to other qualities of micro/monolithic/no-kernels.

However, I do not feel that it warrents further response on my part -- I think my rebuttal, and Mad70's rebuttal to mine, provides adequate material for people to learn a lot about the philosophies of the various camps.  I do feel, however, that a refactoring of this page might be in order, along perhaps with a certain "toning down" of the language to read more like written prose, and not conversation between two people.
<br>-- _(KC5TJA)/6]</span>

<span class="Mad70">[
To _(thin):  I have only responded to an attack (undeserved in my opinion) to _(Fare), I have not initiated it.

To _(KC5TJA):
<ul>
<li>About style: well, this is a Wiki, I think that a conversational style is not so bad.
<li>About Amiga: it is a _(no-kernel) design by accident and I have referred to it only because you have introduced it.
<li>About "toning down" of the language I don't like politically correctness at all (and I'm not fluent with English). However I have no objections to a refactoring.
</ul>
<br>-- _(MaD70)]</span>
