This document describes the T3P, a protocol used by Tunes to transmit data.
Because a system is useful in as much as it can communicate with the external world,
because in its early development phase, it must be cross-developped, hence need to communicate a lot with the external world,
because we want to be able to save individual objects for communication with others or a later ``self'' without having to dump the whole system (
Note that we actually might want to dump a whole system's state as a same consistent object, and actually, we do want such thing be implicitly done at all time, so as to achieve persistency across shutdowns. The point here is that we might want not to, so as to transmit a meaningful part of the system without having to transmit the whole bulk of it that we don't mean and would be noise...),
because of all these reasons and lots more, Tunes needs a robust, well-defined protocol to publish objects.
It just cannot be useful if it stays a closed system, that can talk only to itself until it eventually dies.
Because the protocol to communicate Tunes objects must be able to express all these objects, it must be expressive enough (
Actually, a protocol is a computer language, and the system is only as expressive as its protocols allow.).
Because Tunes is more expressive than any existing system, we know that no existing protocol can fit our needs.
Of course, some existing protocols might be source of strong inspiration...
Internal protocols are specifically left undefined, so that Tunes should freely and implicitly optimize it.
This protocol itself should be have the possibility of dynamically expressing and local optimizations, so as not to be a burden to arbitrarily efficient communications.
Good optimizations should quickly become global, and the conventional protocol will have to evolve with time; but here all modifications will be explicit, by the very nature of an external interface protocol.
The T3P will only consider publication, where objects are sent in. It is an encoding protocol.
It does not handle interaction which can be handled independently by other protocols. For instance, we don't bother here with synchronization problems; we suppose that such information as the size of the manipulated objects are made available by the interaction protocols; and so on.
Of course, high-level interaction protocols can be built on top of an encoding layer, in as much as an encoding layer can be built on top of interaction layers. We have higher-order functors here, not mere static layers.
The T3P comes in flavors, so as to adapt to the underlying media: a random-access flat memory does not have the same constraints as a sequential stream.
The size, endianness, etc, of atomic elements (bytes, words, whatever) also constrains the protocol.
The T3P is still a one protocol, in that adapted flavors are instances of a more generic meta-protocol.
Currently, byte-streams being the one standard low-level way for computers to communicate; hence, the T3P will be have a byte-stream oriented flavor, the TB3P.
Because the stream aspect of data is more standard, and more stable than its byte-orientation, the TB3P will be a byte-oriented specific instance of a more generic TS3P protocol for generic streams.
Another related protocol will be the TA3P, to transmit objects over standard readable ASCII text channels. It may also serve as a low-level text input/output method for objects
If other low-level standards are to be supported, various flavors and sub-flavors will be defined accordingly.
Because we're sure substantial enhancements will be found, as well as possibly bugs, we must have a versioned protocol, so as to be able to seemlessly upgrade it without introducing ambiguities.
Everything afterwards is completely dependent on flavor and version, and may change completely with time.
Below are descriptions for the current versions.
The encodings will be defined in terms of a local context, named the meta-object.
There will be a global registry of standard encoding (sub-)protocols, so that it be possibly to uniquely identify in the exact context in which something is encoded.
Hence, if a machine sees an object with an unknown meta-object, it can unambiguously query a global registry (see <@@ref>TGRthe Tunes Global Registry below), and negociate safe downloading and installation of according handlers.
All that is needed is that meta-objects builtin to basic Tunes distributions (including ``standard libraries'') be enough to connect to the registry somewhere on the Internet (or on CD-ROM, or whatever).
How the Tunes Global registry works is worth a section of its own.
Suffices to say that current version TGR0 is experimental,
with 32-bit UIDs (Universal IDs) as a prototyping simplification.
The Tunes Global Registry, or TGR, will handle Universal Identifiers (UID) for objects that be shared on a large scale accross space and/or time.
TGR0, or ``Tunes Global Registry, version 0'' is the first, experimental, version of the TGR. The design is likely to change a lot until Tunes 1.0, so take everything here with a pinch of salt.
The TGR0 will serve to register coarse-grained modules and protocols, which in turn can contain subobjects.
The address space for this initial meta-object registry, as applicable in the early versions of the T3P at least is 32-bit. This is a compromise to simplify complexity of initial implementations, yet have enough space for some experimentation with semi-automated registry. If 32-bit quickly proves too little, TGR0 might be prompty upgraded to 64-bit. Or we might choose to experiment with address-space multiplexing, with recursive sub-spaces of various (and/or variable) length.
Tunes will have standard primitives to query the TGR0.
As there will be standard TCP/IP ports for Tunes servers to listen, anyone can connect there, and talk using either TA3P or TB3P, then issue TGR0 queries using normal Tunes primitives.
When Tunes is bootstrapped, a tunes.org internet domain will be registered (or something else, if this is already taken). Then, connecting to given ports on meta.server.tunes.org will enable clients over the world to access the registry. The meta-server might actually ask the client to reconnect to a nearer server, but in any case, there will be a known authority to which to connect on the Internet.
The 32-bit TGR0 address space will itself be divided into parts with different registration behavior, and every part can itself be subdivided as necessary for the needs of registration. When the limits of some division are reached, or even before, multiplexing meta-objects will be made available to enlarge the available address space. Or if 32-bit are really too tight, early enlargement to 64-bit may be decided. UIDs are in hexadecimal. dot may be used as an optional separator between UIDs for easier reading.
not to be used, which allows for encoding of local IDs and numbers mixed with UIDs
reserved for Tunes internal usage
reserved for registration by Tunes authorities
reserved for registration by other authorities
dedicated to semi automatic UID registration To be subdivided among registrars.
dedicated to fully automatic UID registration. To be subdivided among servers.
reserved for conservative expansion towards longer UIDs.
not to be used, which allows for encoding of local IDs or numbers mixed with UIDs
This section should describe the beginning of the Tunes-reserved TGR0 address space between 0100.0000 and 0100.FFFF; more UIDs may be allocated in this division, and even more outside of it, but suffices to us to describe enough objects to get the whole thing bootstrapped and enlighten the process: for the exact state of the whole registry, better contact the global server!
unused, for now
Main Tunes objects, such as Tunes itself, the TGR0, the standard portable initial execution environments, the standard fallback encodings, etc.
Core primitives from the initial Tunes 0.x execution context
primitives from the Tunes system access library
primitives from the Tunes 0.x standard library
primitives for Tunes on "standard" 32-bit flat architectures
primitives for Tunes on i386 platforms
reserved for later versions
By the time the needs arise, or when Tunes reaches version 2.0, we'll have understood what is a good encoding for standard meta-objects, and will switch to an enhanced protocol.
Backward compatibility is definitely not a problem, as long as possibly needed conversion routines are freely available. During early phases of Tunes, and until the system stabilize, we'll feel free to modify the design considerably because there will be no user-base to break.
We'll see the design when we're there. Most probably, an expanded version of TGR0, with say 64-bit UIDs, depending on the needs that will have been experimentally mesured. Perhaps v1 might be a conservative extension to v0, if v0 proves successful enough, and such design is favorable to compression.