Next Previous Contents

6. TUNES Implementation

6.1 Will TUNES be Lisp-based?

You'll have to be more specific about your question. What do you mean "Lisp-based"? If you mean built on top of Lisp, then yes, the one Tril is writing now is on top of Common Lisp. Faré also wants to write TUNES starting with a dialect of Lisp. The finished TUNES won't necessarily be like Lisp, but you will be able to run Lisp, as well as many other languages, in TUNES.

6.2 What is the initial plan to bootstrap TUNES?

If you're asking about what I should be writing first as part of TUNES, then I guess the answer, to me, is some open infrastructure for code transformation based on rewrite logic. This infrastructure would get bootstrapped from Common Lisp: a Lisp processor would transform rewrite rules into some Lisp, and then, this same transformation would be rewritten using rewrite rules; macros can help factorize things a lot to minimize this bootstrap. Once the basic infrastructure is bootstrapped, we'll use the rewrite rules to transform a suitable Lisp dialect into some intermediate representation then into some annotated assembly and finally into binary. The Lisp dialect would have some kind of linear Lisp subdialect which would allow for explicit memory management, which in turn would be used to implement memory management (including GC and persistence) for the rest of the system.

So at the end of the bootstrap phase, we'd mostly have some Lisp dialect with rewrite rules; but more than that, we'd also have a code transformation infrastructure that we can work toward making generic, instead of specializing it toward compiling just one Lisp dialect. Actually, we WOULD be specializing it, but not manually; rather, we would try to systematically develop metaprogramming tools that would automate the specialization out of declarative descriptions and specialization hints.

6.3 What are longer-term plans when TUNES is bootstrapped?

Our mid-term goals would be to work out the initial bootstrapped system into such a generic metaprogramming platform that can be used not just to dynamically compile itself into binary, but to do arbitrary meta-level manipulations of arbitrary structures into arbitrary other structures. On the syntactic front, we'd begin manipulating lowly HTML and XML, then jump on to more arbitrary grammars, including subsets of other programming languages, as well as any ad-hoc stuff we need to interoperate with the external world.

This could be the basis for semi-automatically interfacing with code from other systems, or even semi-automatically stealing it, starting with the nicer ones and moving toward more complexity (Scheme, Haskell, CAML, Mercury, Common Lisp, Java, etc., culminating with C; C++ is too horrid to ever salvage its code into TUNES).

On the semantic front, we'd develop (or port) graph manipulation libraries, with systems and metasystems for giving semantics to graphs: computed vs inherited attributes, type systems (abstract interpreters), combination of higher-order modules or rewrite systems; then we can investigate type-directed transformation techniques, deforestation, partial-evaluation, and other well-known algorithmic optimization techniques.

On the heuristic front, we'd develop expertises for declaring multiple implementation tactics for given concepts, then specifying combinations of them as a way to compile code; on the one hand, this would lead to declarative compiler traces that can be used by many metaprograms (invariant checkers, interface extractors, automatic code instrumentators for GC or debugging, etc.). Then there would remain to develop expertises to automatically select best implementation techniques according for a given structure, to cost models and usage constraints (e.g. even for a "same" abstract concept of array, it would choose different algorithms to handle 16KB chunks of real-time data, 1MB of interactive gaming stuff, or 1GB worth of symbol-crunching databases).

We'd then develop some performance introspection technique, so the (manual and automatic) decisions can be based on actual measurements instead of some semi-educated guess of a model. Actually, models are of course both useful and unavoidable, but we should eventually be extracting better cost models from actual dynamic measurements and rough meta-models. Due to resource limitations, in the mid-term, we'd only develop proof-of-concepts and/or kluges for all these, as we need them; but keeping in mind that the goal is to clean them eventually, at the cost of time-compatibility (who cares about time-compatibility when you have the whole-system source to metaprogram into a new coherent version and recompile?).

At the same time in the mid-term, we'd also have to make the system an expedient choice for everyday programming, by interfacing with legacy code: we'd provide outside programmers with a robust persistent dynamic object system that they can use over the network as an "intelligent database", while we'd develop wrappers around existing libraries, to be able to reuse the wealth of existing code (from picture/sound/animation/PostScript/TeX/HTML/whatever viewers to bignum/linear algebra/cryptography/X-window interface libraries to games or foreign platform emulators).

The result of such an effort would be to have a system that could integrate the functionality of Linux in a coherent way, where functionality is wrapped into subprocesses with well-declared semantics, so that the reflective TUNES infrastructure can do whole-system invariant enforcement, automatic robustification of libraries (putting them into isolated subprocesses; intercepting system interaction and doing additional safety checks; detecting failure and restarting them; etc.). This can bring a great incremental improvement to Unices, and lead Unicians to move to TUNES first as a way to better manage their whole systems or just subsystems (instead of having the horrid mess in /etc/* and ~/.*).

In the long term, well, we'd have to fully develop and seamlessly integrate all the ideas and techniques that have been proposed as mid-term goals. Continuing on that trend, we'd get the system to be based on generic declarative descriptions, and to be self-compiled through a series of keener and keener expertises, based on dynamically obtained measures automatically taken according to profiling expertises, using heuristics such as genetic algorithms, constraint propagation (a la constraint-based logic programming), dynamic monitoring to detect deadlocks or optimization opportunities, etc.

Actually, by properly developing the meta-level expertises, we could have a seed proto-AI that would be expert in developing programs, and could be used to incrementally improve on itself until it is able to fully rewrite itself and maybe move toward more AI. Other directions for improvement would be to engineer into the system such things various as checkable formal program proofs, integration of the whole toolchain into both a proof system and a symbolic mathematical manipulation system, use of formal knowledge about programs in program transformations, decompilation of human-maintained low-level C programs into high-level TUNES programs, making the system self-hosted and stand-alone (no more dependency on C legacy), running on bare hardware by stealing or converting C drivers for Unices, compiling to (D)FPGA, automatically extracting man-machine interfaces (GUI-style as well as voice-based, text-based, or anything that suits the needs of the user) from a dynamic combination of program structure, interface declarations, and measurement of actual use, etc.

There are just so many things I'd like to experiment with TUNES! As you see, TUNES does not ambition per se to do things that have not been done in the past. It ambitions to do them in a new way, that hopefully allows them to seamlessly integrate different points of view. My general wish is that in computer science, it becomes true, as it is true in mathematics, that abstract theorems proved once by someone somewhere can thereafter be used by anyone anywhere with finite one-time cost. This is not true due to programs being tied to low-level considerations, and there being no metaprogram to automatically adapt programs from one low-level context to another, even less ones to reason about programs and manipulate them with any kind of dynamically enriched sense of intelligence and obviousness. In other words, there will always be more than one way to do things, but you should be able to prove and the the system should be able to understand that they are about the same thing, and that one way can be replaced by another, depending on what suits the context best.


Next Previous Contents