Conclusions from my OS/Language work (Retro, Forth, etc.) by Tom Novelli THIRD DRAFT - 21 March 2003 I got into this whole business because I was dissatisfied with the operating systems and languages of 1997 -- the complexity, rendundancy, repetitive typing and clicking around. A better, simpler, more uniform system wouldn't be too much to ask. What convinced me that it was within my grasp was Forth. It seemed perfect at the time. Forth! Tunes! Free software for the people! I was still a misguided idealist. Turns out, most people couldn't program worth a shit if they wanted to, which they don't, but they're willing to pay for software that works well enough. Let's be honest: Microsoft is good enough for those folks. So where does that leave us? Myself, I still want a simple "hackers' OS" with good development tools. But in general, user-computer interaction is the one big software problem, and it can be fixed without radical changes. Give people more control for less effort. For example, put each program program or suite in its own directory under /bin. Put support files, documentation and source code in subdirectories, not scattered all over, and search everything under /bin just by having /bin/* in the search path. No installers and package managers, no registries, no mystery DLLs, just unzip or delete programs at whim. GUI programs can be improved. Set aside part of the screen for clocks, music players, status info, etc.; devote the rest to the task at hand. If you've got a mouse, use it, but provide short keyboard commands for everything. Use menus as a learning aid; don't bury anything three levels deep, and use contextual popup menus rather than pulldowns and toolbars. Dispense with long-winded help files; a "cheat sheet" usually says it all and it's not a chore to write. And if something takes too long, don't put up a progress indicator.. optimize it! Things I'd like to see eventually: File descriptions. A modern equivalent of the old DOS Turbo C/Pascal development environment. Simple programs that do one thing well. Coordination between these programs to build complex documents programmatically. Common data formats for vector graphics, equations, document layout, etc. FORTH, GOOD AND BAD At first sight, Forth was just what I'd been looking for, simple yet powerful, one language for everything: operating system, user interface, ad-hoc scripting, major applications, and flexible data formats (like Postscript). Writing my own compiler and hardware drivers was easy, and consisted of mostly assembly language. Writing editors and assemblers was harder; I get sidetracked shuffling things around on the stack, and it's even worse when algebraic formulas are involved. So I thought I'd write an extension to parse formulas and translate them to Forth, and have the best of both worlds. In the course of my parser studies, I came across Small C. The compiler is the size of a simple Forth, inherently faster, and it produces tighter code. The concept is simple: the parser methodically translates infix formulas into postfix, stack-based, assembly/machine code -- basically it translates to Forth! But it only needs one stack because it keeps track of every push, pop, call and return. And it has a few data types, so it knows whether a variable contains a character, integer, or a pointer; therefore, even the simplest C compiler can make optimizations that are impossible with Forth. Forth has a bunch of words (1+ 1- +! @ ! , C@ C! C, COUNT and others) to do what a handful of C operators (++ -- += * &) can do... for example: stacks! For comparison, I did a little experiment: I wrote out Bresenham's circle algorithm in Forth and C, then hand-compiled each (all things being equal except for the number of stacks) and showed it to the folks on #forth, who grudgingly admitted I was being fair. The Forth version was slightly bigger and slower, owing to the extra stack which the 80x86 doesn't directly support. What if we used an ideal dual-stack machine? Since C translates directly to Forth, Forth can never win the race. Forth is an incremental compiler. You start with some core words and add more as you go along. New words are compiled from source code and added to the top of a contiguous "heap" of memory. When you redefine a word, the old version remains lower in the heap, and existing words still use the old definition. You can lop off part of the heap or discard it entirely and start fresh. A multitasking Forth system can spawn multiple instances with separate heaps and a common core. This is somewhat incompatible with the common method of compiling programs ahead of time and linking to shared libraries at runtime, which is more efficient in theory. Forth is well-suited for computers with limited resources, like the ancient minicomputers it was originally written on, or today's embedded microcontrollers. For beginners, it's a good language to start with, and it's also a good place to begin the study of compilers and interpreters. IMPLICATIONS FOR TUNES For small programs, even crazy languages like 'bash' and 'perl' have shown the utility of incremental compilation, but if the primary language supports it while using standard libraries, so much the better. I considered making a "reflective" Forth where words could be retroactively redefined, replacing the old definition instead of adding a new one. This would make it possible to change running programs, for better or worse. Visual Basic supports this, up to a point, then it insists on restarting the program. The simple solution is to break large monolithic programs, like Linux, into independent modules. As an intermediate language, such as the proposed TUNES LLL, Forth is a poor choice. It does little to abstract the differences between hardware platforms, and it prevents some optimizations while adding overhead. The LLL should use data types and only one stack. LISP or Smalltalk syntax would be better, as long as nobody has to read or write it. I've looked into orthogonal persistence, non-hierarchical filesystems, database filesystems, scalable "cube" supercomputers, realtime scheduling: interesting problems, perhaps, but these are things are questionable utility, probably outside the scope of TUNES. Maybe I'll write more on that in the future.