| This article serves two purposes. It is a discussion | | | | Operating systems and applications are beginning |
| about the future of computer interfaces; and it is | | | | to capitalize on what 3D has to offer. The precise |
| a means by which I can purge myself of thoughts | | | | nature of how and where 3D can best be |
| that have been accumulating on this topic for | | | | incorporated is an open question, and a |
| quite a few years. Even if it fails as intelligent | | | | framework to evaluate these questions seems |
| discourse on the first, it will have succeeded in the | | | | appropriate. As a rough starting point it seems |
| second. Previously the title was Where Are | | | | reasonable to divide the attempts into two broad |
| Computer Interfaces Going? but after writing it I | | | | categories: those that are trying to simulate the |
| noticed a significant number of predictive | | | | physical world and those that prefer more |
| passages and decided to be bold and move the | | | | abstract representations. If you'll indulge me, I'd |
| "are". Of course now I feel obliged to add a | | | | like to call these two approaches, respectively, |
| disclaimer. I admit right here, or at least in the | | | | the "Physical Simulation Approach" (PSA) and the |
| next sentence, that I don't know where | | | | "Abstract Representation Approach" |
| computer interfaces are going. I don't know.With | | | | (ARA).Developers in the PSA camp are taking |
| that out of the way, I'd like to start, as many | | | | physical simulations and hanging applications, web |
| interfaces do, with the metaphor. In the 80s and | | | | sites, movies, and pictures on simulated walls. |
| 90s successful interface design and an appropriate | | | | Simulated desks have functional simulated |
| metaphor were taken to be nearly synonymous. | | | | calculators on them. And, perhaps, there is a |
| Although a good metaphor is important, it | | | | simulated sun outside. It's all very familiar and |
| imposes unnecessary and artificial restrictions. So | | | | comes with a nice minimal learning curve.The ARA |
| why is it so important? The best, perhaps only, | | | | camp are working on strange visualization |
| reason is familiarity. Unfortunately, familiarity | | | | techniques to view complexity and patterns in |
| comes at a cost: the shorter learning curve can | | | | large amounts of data. They have general graphs |
| require speed and ability to be sacrificed.Consider | | | | floating around in space with links joining concepts |
| the ubiquitous desktop metaphor. What is more | | | | and words together in arbitrary ways. They have |
| powerful, the abstract construct of a tree, or a | | | | nifty algorithms that filter the salient |
| single flat surface to place your papers on? Well, a | | | | characteristics of large data sets so you don't get |
| tree is. In fact it is so much more powerful that it | | | | overwhelmed. Their attempts are, by far, much |
| is the cornerstone of all modern file systems. | | | | harder to describe with these mere words.In |
| Trees are great, they impose an organizational | | | | practice many attempts will combine aspects of |
| order that is common in natural systems. General | | | | both philosophies. I suspect that successful |
| graphs are, perhaps, too general. DAGs (Directed | | | | attempts at a 3D interface will have to balance |
| Acyclic Graphs) are a good contender; largely | | | | these two extremes in appropriate ways. Objects |
| because of their acyclicness, but also because | | | | in a functional 3D interface should probably be |
| they extend trees in a well defined way. I | | | | represented with models that are familiar, just like |
| suspect that trees are so useful because we can't | | | | the icons on your desktop are often imitations of |
| move backwards in time. Species speciate, | | | | familiar real-world objects. This is a PSA property. |
| languages extend, and software bloats. To fight | | | | On the other hand, tree-based organizational |
| these is to fight the increasing entropy of the | | | | systems would be well advised. Very much an |
| universe.Would it be a good idea not to allow | | | | ARA concept.Text should always be view-plane |
| folders within folders within folders just because it | | | | aligned, as should images. This is one of those 2D |
| would be physically cumbersome, and at some | | | | features mentioned earlier. Images and text may |
| point impossible? Probably not. Do icons have a | | | | be scaled, but they should not present |
| real-world counterpart? Not really. Metaphors | | | | themselves at an angle. Vertical and horizontal |
| should be, and have been, taken only so far.So | | | | edges need to remain vertical and horizontal. Of |
| what does the future hold? Will interfaces be 3D? | | | | course, these features are trivially present with |
| Will we be stuck with rectangles forever? I think | | | | your desktop interface as well.And there's an |
| it's reasonable to say both have their place. People | | | | important lesson: build on the backs of giants. The |
| on the 3D side think that we humans see, work, | | | | desktop UI is successful for a reason, not simply |
| live, and play in 3D. We don't. They say they can't | | | | because it has a familiar analogue in the physical |
| wait until there are fully 3D monitors that you can | | | | world, but rather because it behaves in that same |
| walk around. Why? Our retinas, as well as birds | | | | useful way that real desks behave. It takes |
| whose eyes are plastered on the side of their | | | | advantage of a well-established ability; spatial |
| heads, are two-dimensional surfaces. Birds have | | | | memory. You put something down and it stays |
| flatter vision than we do, if not as Euclidean, | | | | there.Useable interfaces need a certain amount of |
| because they don't have the benefit of the tiny | | | | persistence in their structure. Having objects stay |
| bit of 3D depth perception a predator gets by | | | | where you leave them is one good way to |
| overlapping images. I've heard graphics | | | | achieve persistence. Placing objects manually, |
| programmers explain that their 3D scene was | | | | whether on your desktop or in a 3D environment, |
| being projected onto a flat 2D screen and so it | | | | takes advantage of spatial memory. We can |
| was no longer really 3D. But consider this: | | | | remember, in context, where we've left hundreds |
| everything you see in this world is like that. It all | | | | of objects (notwithstanding car keys; they get |
| gets projected onto our flat retinas. We just have | | | | moved around too much). You probably know |
| really big brains. A 3D scene is constructed in our | | | | where your camera is and where the light |
| mind regardless of whether what we're viewing is | | | | switches in your home are. By positioning objects |
| on a flat computer monitor or in that | | | | manually you can give them some context; |
| nether-world known as real life. In fact, most | | | | perhaps by placing pictures of your family to the |
| brains do a decent job of scene construction | | | | left, and panoramic vistas to the right. Contextual |
| even with one eye closed. From 2D to 3D. | | | | clues help you remember.I've heard the assertion |
| Impressive!People on the 2D side think that we | | | | that adding a single extra dimension doesn't buy |
| humans see, work, live, and play in 2D. We do, | | | | you much organizational power and that the |
| after all, have flat retinas, like playing tennis on flat | | | | added navigational complexity isn't worth it. Others |
| tennis courts, and eat dinners from flat plates on | | | | think that we need an n-dimensional space to do |
| flat tables. But we don't live in 2D. Our brains are | | | | a good job. Aside from the obvious observation |
| really big. 1.3 litres big. More than enough dendrites, | | | | that we seem to exist in a macroscopically |
| axons and other brain-things to contain a nice 3D | | | | three-dimensional world (macroscopically was |
| representation of the world we live in. Clues to | | | | added just to keep any |
| build the scene abound: motion, foreshortening, | | | | physicists-who-may-know-better reading) and are |
| and the aforementioned depth perception.The | | | | therefore good at 3D manipulations, there is |
| truth is some things are better in 2D and some | | | | evidence that the jump from 2 to 3 dimensions is |
| 3D. Writing a letter? Use a desk. Put a flat piece | | | | of a more fundamental significance. If you draw a |
| of paper on it. Want to file that letter away? | | | | bunch of dots on a piece of paper you will not be |
| Wouldn't it be cool if you could just let it hover in | | | | able to draw lines joining the dots in all possible |
| some large 3D organizational space? Here's what I | | | | configurations unless the lines cross (given some |
| think. | | | | sufficiently large number of dots. I think 5 might |
| Text: 2D | | | | do it). However, once you hit three dimensions, all |
| Reading and Writing: 2D | | | | configurations are possible without crossings. |
| Organizing and Grouping: 3D | | | | Adding a fourth or fifth doesn't have any further |
| Visualization of Data: Depends | | | | beneficial effect. Admittedly there is some |
| It has occurred to me that 2D representations | | | | hand-waving going on here; but the result has |
| should be considered a feature of an interface. It's | | | | implications for some possible interface designs; |
| beneficial that text documents are lined up nicely | | | | and it points to using three dimensions.So why |
| for you in a window. If head-or-eye-tracking | | | | haven't interfaces changed much in the last 20 |
| hardware were more widespread, we'd have | | | | years? One possibility is that the desktop is in |
| software that could compensate for (single) users | | | | some way an optimal representation. More likely, |
| who are not directly in front of their screens. | | | | however, is that it is simply a functional |
| Imagine looking at your monitor from an angle but | | | | representation; no need to change when change |
| still having the text of this article appear flat. That | | | | takes effort, right? We expect to be able to sit |
| would be a pretty neat feature (on the other | | | | down in front of a new interface and immediately |
| hand, it might just look strange and make you | | | | be as productive as we were before. We have all |
| sick; hard to tell without trying it).Because the | | | | learned to use the desktop and menu-driven |
| input is essentially 2D, I predict pure 3D imaging | | | | interfaces because we haven't had a choice. It |
| devices will prove to be a novelty even if the | | | | has taken time; just as learning to read and write |
| enormous bandwidth problems can be solved. A | | | | took years when we were younger. Even the |
| graphics card that draws a 480x480 pixel scene | | | | keyboard and the mouse, although perhaps easier |
| at 60fps would take 8 seconds to update a | | | | than writing, have taken time and effort to |
| 480x480x480 cube. Yes, I understand this is a | | | | master. New interfaces will face the same hurdles. |
| vast simplification. Somehow restricting rendering | | | | Their designs will need tweaking to reduce the |
| to the surfaces of an object might help, but it | | | | learning curve as much as possible. The users of |
| sounds tricky. Regardless, the same or better | | | | these new interfaces will need the patience to |
| effect will be achievable by feeding a couple of | | | | develop efficient usage patterns; and the |
| 2D images to each eye. Technology that takes | | | | interfaces themselves will need to be entertaining |
| this approach will be more successful. Devices | | | | enough to mitigate the patience required. All these |
| that project images directly onto the retina seem | | | | efforts will yield interfaces that are not only more |
| like a reasonable approach; along with any tracking | | | | enjoyable, but faster and more useful.Tristan |
| technology that may go with them.The next 10 | | | | Grimmer is technical director at Upper Bounds |
| years will be a transitional phase for interface | | | | Interactive Inc. Previously a video-game |
| design. 3D rendering technologies already have a | | | | programmer, and much earlier than that a young |
| stable home in the entertainment, video game, | | | | boy trying to get his Vic-20 to accurately |
| simulation, and design sectors. Although 2D | | | | compute Pi, Tristan now spends his time working |
| interfaces have dominated everything else, I | | | | on Tactile 3D in an attempt to rid the world of |
| expect we will start seeing more 3D incursions. | | | | rectangles. |