Wednesday, February 18, 2009

Data-centric command line user interface

The typical graphic user interface of computers has not changed very much since the introduction of the WIMP (Window, Icon, Menu, Pointer) interface. You see it throughout X on Linux, Mac OS *, and on Windows. Since its conception at Xerox in the early 70s it's surprising that the core properties of WIMP have not really changed. Layers have been added (e.g. the task bar) and various facades (we have novelty sized buttons on transparent windows), but the general direction at which the major desktop GUIs have been developing in appears to represent a comical race you would see on a cartoon; when one GUI finds a way to jump ahead (e.g. composition (yet no one has invented rotatable windows!), or sub pixel text) the others think implementing that idea is the only way to catch up to it, only to find that when they add something innovative the other GUIs do the same. The world of desktop GUIs is stuck in a game copying each other rather than branching off into their own direction.

Your typical desktop GUI interface has not really changed. These GUIs represent the program's output as a movable square rectangle called a window. These windows typically (with some exceptions) contain a boarder around them, a titlebar above the window that can be dragged to move the window, minimise/maximise/restore/close buttons on the titlebar, and some windows can be resized by dragging their lower right hand border. These similarities cause virtually every desktop GUI environment (with some exceptions like borderless window managers) to be fundamentally the same.
I have a question that I would like to ask;

Are there any alternatives to WIMP that can still provide the same (or greater) level of flexibility and productivity in a desktop multitasking environment?

Most people would answer no, but I believe we are to use to working in WIMP that it is hard to imagine outside of the box. Programs do not need to have their own visual form of output, and I’ll explain this a little later on with an idea I have.

I have been looking into alternative user interfaces, and there are two fields I see that have potential; modeless environments, and zoomable user interfaces (I have a hunch that something big is in store for ZUIs). Modeless environments, like Archy, are still highly experimental, and it’s a predominantly single-tasking environment. As far as accessibility of information goes I think Archy will be an interesting contender to keep my eye on though I think it will lack in productivity when it comes to proving itself. I haven’t actually seen a ZUI that hasn’t been domain-specific (most commonly a file browser or a mind map) and I wonder what route will be taken (I think it would be cool to zoom into your application’s icon to launch it and zoom out to close) but I still see WIMP living on, just instead you’re zooming in and out of your windows.

I accept the possibility of the answer to my above question to be ‘no’, but a simple no will not provide closure to the question when the alternatives have not been fully investigated. I can see WIMP’s evolution slowly progress with the introduction of the ribbon in Microsoft Office to replace many of the traditional menus. I congratulate the team who came up with the concept, and I can see it’s a path in the right direction, but it’s also interesting to see users’ reaction. At first I, along with a lot of other people I apparently discovered when searching through online forums, found it uncomfortable and had nostalgic memories of the old interface. It’s not until you get use to it that you find it more productive, which brings me to wonder that if a better replacement for WIMP came people would be too nostalgic and comfortable in their old environment to want to swap.

Now back to what I mentioned before, applications don’t have to be “applications” with their own viewport as we think of them today (one exception being full-screen games). Imagine a persistent user interface where applications are merely thought of as extensions to your environment rather than self contained programs.

I think for user interfaces to progress to what I am conceptualizing in my head we must take a data-centric view of the computer and the interface. The core essence of the interface I am seeing allows you to create, view, manipulate, and share data in a persistent interface regardless of the content and type data. You’re data could a word document, an email, a song, or even a web page. Programs are just extensions you load into the environment that enable you to work with a new sort of data (e.g. a spreadsheet program allows you to work with spreadsheets). When I try to visualize it I get the image of a Mathematica-style environment which integrates graphics with graphics inside of a command-line interface. But there is still the obstacle of allowing you to move your data around the screen, which eventually brings us back to a traditional WIMP interface. The only way I see this sort of interface happening is as an alternative shell, rather than the sole interface of a computer.

I think it is unnecessary to limit future GUI designs to the desktop metaphor. With tomorrow’s generation of users growing up with today’s interfaces, I don’t think there is a need to keep interfaces ‘simple’ to sacrifice productivity (without making them overly complex). My mind is debating both sides in my head; to WIMP, or not to WIMP.

Tuesday, January 20, 2009

New Blog

While I'm looking for employment I'm working on a new game framework called Mercury Game Framework 2 (see the original MGF on my website).

The main goal of MGF2 is to create a completely platform independent framework which I can call my own. Since I find writing graphics engines fun this framework will mostly be a graphics engine though it will breach into other areas (input, audio, etc) just not as in depth. So far it just runs on Windows, though in the future I'm hoping to port it to Linux and Xbox 360.

I want to make the framework future-proof by making it completely scalable. So that means everything has to be thread-safe, and it should distribute itself across cores automatically (be it 1, 2, or 32 cores) making as much use of each core as it can. Right now the code for doing this is in there, though I haven't stressed tested it yet.

I want the framework to be as flexible as possible. Accessing files will be done through interfaces (so the file may be stored in memory, compressed, or accessed over a network), and it should be possible to provide a custom memory allocator (who knows, maybe it will run on an architecture that requires this).

Just a minor thing that has always annoyed me. Multiple string formats! As far as I know, std::string doesn't provide anything for converting between ASCII/Unicode (which is actually really simple to do, but something I have always wanted it built into an = operator). So I wrote my own string class to overcome this problem:

String8 str = String16("my widestring");

With "String" (minus the size at the end) mapped to the compiler's default character size. Please, if you're writing your own string class, take this into consideration.

There is no fixed-function pipeline, and shaders are written in CG. Right now this ties down the framework to only platforms that support D3D or OpenGL, but it is a reasonable payoff for the flexibility this gives my engine. I personally don't see the framework running on anything that does not support shaders and I haven't planned for it yet, but if I really needed it to be so I could design a failsafe material system.

I use XML a lot aswell. Here is a vertex in my model format (I have an exporter than converts .X/.FBX meshes to .xml):





At first I thought this would make the file huge, but it's smaller compared to an equivilent model in .fbx format. If size does become an issue I could make a binary XML format. All resources are defined in XML (though binary data like textures are still stored in their native file formats, but the definition stating their resource name and file are in XML). You can have as many meshes, textures, sounds, materials stored in a single XML file, and I've designed it to be simple enough for a program to hook in their own resource loader (e.g. to load weapons, vehicles, or a map) from the same XML file.

The framework still is very young, though all the above I have implemented already. What I'm working on now are loading textures. Lighting isn't in yet, neither is any input code. I have a sample project (right now it's Pong, and when Pong is done it'll be Minesweeper) so I have a realworld idea of what the framework requires in order to create a working game (so I don't leave anything out in the early design, and I don't go off into tangents on something unneccesary).

I also have big plans for an online game (it's not very art heavy, nor does it use overly complicated logic, but I have a gut feeling it will be some-what popular) I plan to build with this framework (that is until I'm employeed and don't have time to work on it anymore).