Weekly Retrospect

John Underkoffler points to the future of UI

John Underkoffler was part of the team who developed the visuals for the movie, Minority Report, which is where everyone gets their idea from about what a computer of the future, utilizing Natural User Interfacing, would look like.

What's remarkable is the John and his team developed these visual as if they were actually undergoing Research and Development of this technology which is why so many developers today still turn to Minority Report for inspiration and direction when it comes to NUIs. It's obvious that iPhone's gestures are greatly inspired by this film which is why the phone is so simple to use. We are all already aware of how to use these (then) non-existent interfaces.

John claims this UI will be in every computer sold in 5 years time and within this decade we can expect this technology built into every monitor in our homes; even in our walls to work with projectors.

Input Agnostic Design

With Weekly Retrospect I set out to design a UI that could be interacted with in multiple ways. The current version supports keyboard, mouse and with a webcam and an application it can even support gesture interaction. Swipe and tap gestures (iPhone and iPad) can easily be supported once the onboard browser opens up access to it's gesture APIs which means that this UI would work seamlessly with iOS devices.

The simple commands required to navigate this UI means it can also easily translate to remote controls (such as the Apple Remote or any Universal Remote) requiring only directional buttons and function button (for triggering zoom mode) to function. In the same way. this UI will be able to be controlled using Playstation/Xbox360 controllers and even through voice commands ("Next Article", "Previous Issue", "Zoom In" or even just "Up", "Down", "Zoom".)

Unlike a traditional website which is designed specifically to be used with a mouse (often not even both a keyboard and mouse) WeeklyRetrospect's UI is based upon abstract commands that only requires an interpreting script that can interpret keyboard commands, mouse clicks, gestures or even spoken words into these abstract commands and manipulate the UI.

Abstracting the commands allow for multiple forms of input for one design, only requiring a script that translates the various input commands into commands that the UI understands instead of creating a new UI for each input type.

Wired App for iPad

iA on Wired's iPad App

Let's make this clear once and for all: at the current surface and resolution of the iPad, multi column layouts for long screen texts are sentimental nonsense. And the more columns you use, the worse it gets.

— Just Like a Paper Tiger, Oliver Reichenstein

He's right. That's why Weekly Retrospect is designed at comfortable line lengths, revealing only more columns when there is excess screen real estate. The revealed columns is not meant for consumption, they are a visual cue that there is more content to the right. On the iPad, this interface would scale up and most of the screen would be the one column (revealing only a little bit of the next column to solve the following problem.)

Gruber on Wired

They're using both axes for pagination — you flip left-to-right to go between stories, and you go up-and-down to go through the pages of a story. One problem is that in many cases, especially with multi-page ads, there's no visual clue to tell you when there are more pages underneath the current one.

— Wired Magazine, John Gruber.

It's interesting to note that Wired have also grasped both axes for navigation. Keep in mind that there is no "between issue" navigation on the Wired app, this involves closing the app and opening a new one. Wired have opted for the horizontal axes to be between articles and vertical to be pages within articles. The Weekly Retrospect UI can easily be adapted (having articles run vertically like our issues and pages within articles horizontally as they currently do.)

I think that Weekly Retrospect's use of both axes is superior to Wired's choice. It 's obvious when there is more content in the current article (with the preview of the next column being a visible cue of where to go next) and when you've reach the end of an article. Having articles separated vertically also makes sense because the first "page" can be used much like a contents page is in a novel, used to skip between articles/chapters. We are much more used to this being a vertical motion than a horizontal one, as we are more inclined to read the "next page" horizontally than vertically.

Aladdin

People prefer living life, believing in the little myths that we believe makes life makes sense. This week I watched Aladdin, recording all the little myths that I saw that are prevalent throughout Western story-telling.

• “A diamond in the rough” — people are more than they seem
• Good vs Evil. There are bad people who only do bad things
• Only Chosen one may enter the cave. "Less than worthy"
• Poor people can steal, rich people are evil.
• Bad guys are dumb.
• Poor are good of heart — gives all he has to the poorer children
• You get married for love
• Princess runs away from responsibilities because she wants a normal life
• The grass is always greener on the other side
• Greed is every person's downfall
• Friends are ALWAYS loyal
• Love is worth an eternity of servitude
• Riches bring corruption
• All stories end happily
• Woman are helpless

These are just the ones I noticed, I'm sure there are many more that I was unaware of.

The Good Wife

What intrigued me about these myths were that when I watched The Good Wife, in the episode, they had a client for was being prosecuted for murder which they discovered was actually an accident. The prosecution were conflicted on whether to argue the fact that it was an accidental suicide (the truth) or whether they should implicate someone else to get their client off on reasonable doubt.

People don't want to hear the truth. They want to hear something they can believe

The lead attorney said that people didn't want to hear the truth, they wanted to hear something they could believe and that people believed a story where there was a villain involved and not just one of pure chance. He said that if he didn't paint the picture of another villain then the jury would choose their own, and that might just be his client.

In the end, they prosecuted with the truth, which did win out in the end, confirming yet another myth — that the truth always wins out in the end.

The truth prevails

Justin Bieber, not so innocent

A lot of things in society that have traditionally been seen as natural but has since been deemed as politically incorrect still lingers on in our daily lives. The other day when I heard one of Justin Bieber's songs I sniggered at what I heard. Justin Bieber is known for being the a cute romantic, his lyrics are all about falling in love and going to the ends of the earth for his beloved—despite only being 15 when he wrote them.

Here's the type of lyrics he is known for:

How many I told you's and start overs
And shoulders have you cried on before
How many promises be honest girl
How many tears you let hit the floor
How many bags you packed
Just to take 'em back tell me that
How many either or's but no more
If you let me inside of your world
There'd be the one less lonely girl

"One Less Lonely Girl", Justin Bieber

The perpetual idea that everyone will one day find their princess, their knight in shining armor, who is perfect for them and will give them their happily ever after.

My prized possesion, one and only

What caught my attention was the following line, in his song "Favorite Girl", My prized possesion, one and only. We all know that he there's nothing in his life more important than his favorite girl but the mere fact that he's comparing her to his other possessions should be a little bit insulting.

Not to mention the fact that he's implying he's got more than one girl, she just happens to be his favorite. Another myth that is so prevalent today, boys are players and girls are sluts

I went to look at some of his other songs and what he says that we just don't hear.

Chorus

Only if you give, give the first dance to me
Girl I promise I'll be gentle
I know we gotta do it slowly
If you give, give the first dance to me
I'm gonna' cherish every moment
'Cuz it only happens once, once in a lifetime

Part of Verse

Everybody says that we look cute together
Let's make this a night the two of us remember
No teachers around to see us dancing close
I'm telling you our parents will never know

"First Dance", Justin Bieber

Not so innocent now, is he?

We all carry baggage.

As we grow up and move through the world we are constantly creating associations between the definitive and the abstract. We constantly increase our library of recognizable signifiers, our baggage, which influences our reading symbols in the future.

Although each person's library of connotations differs, the majority of our libraries are extremely similar. It is this shared database that allows the communication—specifically the visual communication—industry to exist.

There's an idea that we all share, that of the exotic. Which is often used by marketers to try and sell us a product that we should think we want. This product is normally one that allows us an escape from the mundane and banal—in other words, a holiday.

It's extremely uncommon for advertisers to make us aware of these connotations that we have that allow them to sell products to us, so when one company does, the world took note.

Old Spice Advert

The man, your man, could smell like.

The advert for Old Spice takes some of the myths that we all hold true and extrapolates them to the extreme. A perfect man, “The man your man could smell like” is presented in the advert that comments both on the stereotypical man that every woman wants and the mythological world that these ideas are born from.

The ad (shot in just one take) has 3 distinct scenes — in a bathroom, on a boat and on a beach. We seamlessly switch between these 3 scenes in a manner that shows just how fragile this reality really is. The bathroom is not really a bathroom, just 3 walls on a boat—a set, and the boat we were we are on is not really at sea, its just an illusion.

This all just to do with what’s happening in the background, which many people may not notice or just excuse as special effects (in reality the only special effects are the diamond waterfall out of his hand and the rising Old Spice bottle from his palm.) What everyone else will definitely notice is exactly what the perfect man is saying.

Look at your man, now look at me, now back at your man, now back to me; sadly your man is not me. The advert is making an explicit comparison between your man and the perfect man, and at the same time acknowledging the perfection is unobtainable.

It’s an oyster with two tickets to that thing you love. A line that suggests that even the perfect man doesn’t really care what you think or what you like. He’s willing to keep you happy but really, he’s not interested. The tickets then turn into diamonds, supposedly a woman’s best friend.

My favourite part of this advertisement is the causal line at the end, I’m on a horse as the camera zooms out and we see that the perfect man has managed to find himself on a white horse. This does two things, it connects with the myth of the knight in shining armour or the prince riding a white horse but it also makes a comment about the bizarreness of advertising today, and that although anything is possibly in advertising today, it doesn’t always make sense in reality.

Where do typefaces get their voice?

Why do we treat certain typefaces in certain ways. Why are serif faces regarded as elegant and sans-serif typefaces commonly associated with the future, technology and progress? Are there inherent design details of these typefaces that embody these traits or have we learn to associate these ideas with the typefaces based on the context they were originally used in.

Gotham, Optima and Helvetica

This typeface was thrust into proliferation due Scott Thomas' choice to use it as the official type of choice for the Obama campaign. It is said to be the result of American Modernism though I think that in some ways it reminds me of the type used on Communist propaganda (which was albeit more condensed.)

The connotation, I ponder, is one born out of contrast between Hitler's fascism (then the Fraktur, today the serif) and Stalin's communism (the almost geometric sans-serif.) I believe examples like these strong visual languages forever tainted our understanding of type. Our understanding of these ideologies influenced our reading of the typefaces. Fraktur/Serif type is deeply rooted in traditional, which is generally of a capitalist/fascist background whereas the reaction to this, both politically and visually, was communism which focused on the people and released them from the shackles of fascism (all those serifs). The choice for Obama's campaign to use Gotham vs McCain's use of Optima, or in fact, most other Presidential Candidates' use of serif typefaces, is rooted in the visual political history more than the forms of the glyphs.

McCain's choice of Optima suggests a world becoming more liberal. Although the differences between Optima and Gotham are stark (in that Optima, being Humanist, seems to have deep Roman, read traditional, roots) they both are technically sans-serif typefaces. Both liberal and conservative candidates chose to distance themselves somewhat from traditionalism, often associated with serif typefaces.

Hoefler and Frere-Jones claim that:

Political typefaces have a way of being chosen because they underscore (or imagine) some specific aspect of a candidate, working hard to convey "traditional values" or "strength and vigilance," or any number of graspable populist notions. The only thing Gotham works hard at is being Gotham.

In other words, they believe that Gotham is an American Helvetica, which I believe may be true for other applications but not in the case of politics. In this case Gotham was chosen because it was The People's Typeface, as opposed to the Traditional Typeface.

I believe that H&FJ make this clear when they speak about Gotham in a context other than Obama's use of it. They say GQ had a dual agenda of wanting something that would look very fresh, yet very established, to have a credible voice to it, Here they admit that Gotham did bring a voice and it's the key phrase "fresh, yet very established" that I believe would lead a design team for a man who's policies have been criticized of being socialist, to choose this typeface.

H&FJ claim that Gotham is the American Helvetica, Helvetica being the Swiss neo-grotesque sans-serif that was designed by Max Miedinger in 1957 and devoured by the modernist community who were rebelling against the decorated, embellished and overworked design of the 19th Century. They saw things like serifs and stroke contrast as superfluous additions that complicated the form and that the type should be whittled down to only the forms necessary for function.

Garamond, Baskerville and Rockwell

If Gotham seems to represent a culture that wanted to set themselves apart from Traditionalism then Baskerville, Caslon and Garamond represent that traditionalism. Garamond was one of the first typefaces created as movable type, which is what enabled the first printing techniques. As these techniques were expensive at first the only people who could afford to produce printed works (in these typefaces) were those with enough money. Of course a capitalist model existed at the time and allowed individuals or organizations to gain enough money to print. For this reason, we also associate elaborate handwriting (with gratuitous swooshes) with grandness; because the only people who could afford scribes were the rich.

As printed books became the main method for established organizations, like Universities, governments/monarchies, churches and the like to distribute their ideologies and teachings, the typefaces used became associated with establishment, dependability, trustworthiness and formality. Later, typefaces like Caslon and then Baskerville would try to modernize these serif faces by increasing contrast (then possible due to newer technologies.) These were Transitional Serifs.

As new tried to differentiate themselves from old we see visual progress. Didot continues the exploration Baskerville started of contrast between stroke widths (and takes this much further.) Didot is part of the modern Serifs.

The final development in serifs is—as we can expect—a reaction to the high contrasts of the modern serifs. We find this in the low-to-no contrast slab serifs such as the popular Rockwell and the contemporary Archer (which, trying to be new, adds the cheeky ball terminal to contrast with the neo-grotesque block serifs.)

Organising Type by Class and Political Ideology.

In the table below I've loosely organized the different types of type into social class and political leaning. Generally the more higher affiliation to political leaning you are trying to appear, the more extreme your type choice would be. Gotham is more socialist than Helvetica which is more Socialist than Optima. Times New Roman is traditionalist but not as much as Didot which pales to Baskerville. The idea of classes apply in a much greater more in a traditionalist society than a socialist one who try to abolish class.

Below you can see typefaces on a scale from most traditional to most modernist.

I’m tweeting from the treadmill. The treadmill is a content creation device.

@fraying

Calling the beautiful, amazing, brilliant things people create online “user-generated content” is like sliding up to your lady, putting your arm around her and whispering, “Hey baby, let’s have intercourse.

Derek Powazek, “Death to User-Generated Content”

Yet with the “iPad isn’t for making content” crowd, you wonder what they define as content. Twitter, apparently isn’t. Neither is writing, data-crunching, or presentations (the actions offered by the three apps Apple created itself for the iPad — all, it’s worth noting, content-creation apps). Music, out. Drawing, out. Blogging, the very format that these critiques have taken? Out. The iPad lacks a camera, yet there are photo editors for it, none of which apparently create content either. I would expect video editors to emerge shortly (probably from Apple to start), audio editing is already on the iPhone, so it’s sure to make the move (and already runs, just tiny). Is that content? Probably not. I could continue, but at some point I’m just duplicating the App Store catalog.

— Daniel Sinker, “Creation Myths: What the Argument That the iPad’s Not for Creating Content Really Tells Us”

This is why the umbrella “content” term is bad. Sure, you won’t be writing a novel on the iPad (well, probably not but who knows) but you can certainly create other forms of content.

Let’s not throw the bath-water out with the baby.

Where once it was possible, even natural, for design to happen concurrently with editorial creation, now it simply cannot keep up. The pace is too rapid.

— Khoi Vinh, “Jobs Saves?”

The Designer’s Diet

The diet of a typical designer is low in in-depth content and high in inspirational lists, tutorials and freebies.

I hate this about the web, I want valuable content. I love dissecting case studies and hearing WHY designers did something. To me the how is not important, it’s the what and why.

I want to see more of this in-depth analysis of design. Specifically I want to see The National Grid brought online as well, because as I understand it, Jonty and Luke started The Grid because they saw a lack of this type of design commentary.

Not all content is created equal

I’ve really enjoyed and hated working on Weekly Retrospect for just over the past month or so. The idea is that I collect throughout the week everything that interests me right here on this tumblog and then curate a weekly publication at WeeklyRetrospect.com.

This week has been tough, last week was the iPad Week and it seems like this week is iPad Week Continued. The release of this device, for one reason or another, is simply huge and no-one can stop talking about the device and speculating about how it will change our lives.

At one point I gave up and decided that this week I would not release an issue of Weekly Retrospect — I wanted each issue to be it’s own topic and have the content within that issue organised by me in whichever order I thought made most sense and told the narrative I wanted to tell. Having a second week of iPad didn’t seem to work well for me and although I considered the idea of adding to Issue 5: iPad Week instead of releasing another issue I felt this didn’t meet the standard either.

I’m still torn whether I should go back and edit Issues, I guess that’s an advantage of online media but if I do that I need to solve the problem of how do I let people know an issue has new content and then how do I manage their experience of consuming that new content without taking them out of the curated linear reading experience but also not boring them with content they’ve seen before. I still have to figure this out.

Luckily, Derek Powazek got very, very worked up about the fact that a lot of people were saying that the iPad is not for “content-creators” and pushed that further to conclude that the fact that we have latched onto the use of the term “user generated content” is doing an injustice to the works created by these people. Whether it’s articles written by writers, photos taken by photographers, poems by poets, songs by musicians, reviews by critics and so forth. By reducing the world of “stuff that people make” down to “content” created by “users” we are doing a huge disservice to those people creating works of all kinds.

Suggesting that a tweet about your cat and a carefully crafted haiku is much the same by referring to both as “user generated content” is not fair and representative of the value of them. Despite both coming in text form (fitting within 140 characters) they both have extremely different amounts of value and this value shouldn’t be ignored because we’re tackling the latest buzzword.

In this week’s issue of Weekly Retrospect I've covered the fact that not all content is created equal.

I asked him if he would come up with a few options, and he said, ‘No, I will solve your problem for you and you will pay me. You don’t have to use the solution. If you want options go talk to other people.’

I love this video because it’s about two men who get it. They understand the power of specialisation and they can trust someone to do their job. I will solve the problem for you, if you want options go talk to other people.

But I'm telling you, the multitouch screen/software makes it very, very different from a laptop, and the screen size makes it very, very different from an iPhone. It's something entirely new. So yes, if it appeals to you, you'd have to buy it in addition to your laptop or iPhone.

—David Pogue, “Apple iPad FAQs”

Spatiality and NUI

I’ve written before about the differences between direct and abstract interfaces and how these differences can confuse users who aren’t used to abstracting their understanding of reality. The iPad, albeit wonderful at many things, fail to understand a concept very similar to this:

Let's say you want to open the App Store on your iPad, and you know that you've put this icon at the bottom right of your apps. Turning the iPad rewraps the positions of your icons. The App Store now suddenly jumps to the middle of the second row:

While Lukas Mathis provides a few different alternate behaviors, I agree with this one most:

Another solution would be to simply keep the applications positioned as they are, and only turn the actual icons and labels.

I believe this solution is strongest because, as Lukas describes:

While the App Store is now no longer at the bottom right, it's still in the same position. What has changed is the user's perspective, not the arrangement of the icons. Humans are dealing with changing perspectives all the time and should still be able to identify individual applications based on their position.

One of our abilities that can be taken advantage of here is the same ability that allows us all to sit around different sides of a Monopoly game-board but still have the same understand of the game. How confusing would Monopoly be if after every person’s turn the game board reoriented itself to the current player?

Realism Requires Experience

Luke brings up an important point about Natural User Interfaces.

Product’s like Microsoft’s BOB operating system, Southwest Airlines’ original Web site, and even Apple’s eWorld made poor use of “realism” by forcing people to learn from scratch. No one knew what was located in BOB’s house nor where anything was in BOB’s house because they had never been there. Nobody knew their way around Southwest’s ticket counter because they had only encountered one if they bought a Southwest ticket at the airport. Same with eWorld -it was “realism” without the reality. In these circumstances, it’s hard to exploit people’s existing skills because they have none.

So what’s the right way?

In contrast, Apple’s early iPad apps use objects many people are familiar with as the basis for their user interface: bookshelves for accessing and storing books; address books for accessing and storing contact information; and more. These application designs take advantage of skills people actually have acquired through years of interactions with the real world and highlight how physicality and heightened realismmay, in fact, lead to more usable designs.

I think Apple’s interfaces get out of the users way and teach them that their applications are for content creation and manipulation, and not as is the case here, about interface interaction.

— Michael Heilmann on NoScope

Rightly so, this is a classic case of abstract vs. direct manipulation. The iPad pages app lets you directly manipulate the content in a much more natural way. Word forces you, still, to decipher icons whose meaning may be decades outdated.

Icon for the Save button is still a floppy disk, despite the fact that Apple hasn’t sold a machine with a floppy drive for a decade.

—John Gruber, “Leaked Screenshots of Microsoft Office for Mac 2011”

As developers, we want to maintain a pool of testers, not devices that they test on. Devices are ephemeral: they change as new hardware is introduced and replaced. The thing that remains constant are the people who test our products.

— Craig Hockenberry, “UDID not”

Craig talks about Apple’s Ad Hoc Testing allocation being device centric and not people centric. Although not user experience design related it is system/process design-related and again comes down to the fact that technology changes quickly, people don’t. We need to stop focussing on specific technologies and more on design for people.

Our goal isn’t to create new technologies. I mean, that’s what I do, but our goal is to create new experiences.

—Dr. Richard Marks on Episode 7 of The Engadget Show about developing the Playstation Move.

A continuance of the theme to develop for people and not technology/devices.

Is the user always right?

Recent fuss has been made about the Bloomberg Terminal and how it’s users refuse to use anything other than the current, complex and terrible design.

Simplifying the interface of the terminal would not be accepted by most users because, as ethnographic studies show, they take pride on manipulating Bloomberg’s current complex interface. The pain inflicted by blatant UI flaws such as black background color and yellow and orange text is strangely transformed into the rewarding experience of feeling and looking like a hard-core professional.

The Bloomberg Terminal interface looks terrible, but it allows traders and other users to pretend you need to be experienced and knowledgeable to use it.

—Dominique Leca

According to Lucas Mathis, this type of reaction from users is acceptable:

If your users reject an improved user interface, you need to start out by figuring out exactly what motivates them to prefer the more complex solution.

But I’d like to question, is it really? Making jobs more difficult in order to protect the people in them — is that a good enough reason not to improve user interface, not to make things easier. Is it really about protecting jobs or a sense of self-worth or is there a way users can feel like they aren’t being treated as imbeciles and still make their lives easier and the work more productive.

My iPhone is not as complex as Android devices but I love it and as a geek, I’d hate to think that I’d ever give up control over my devices but I have, very much so. But Apple have figured out how to make me like it.

They didn’t listen to me, or the rest of their customers: they knew what was right for us (what we needed), and gave us that instead of what we thought we wanted.

“Notes on Improvisation and Design” by Liz Danzico

Great talk by Liz drawing comparisons between music and design and more specifically the analogy of the more structured classical music (where the emphasis is placed on the creation/composition of the original work) compared to the more accessible Modal Jazz (where the emphasis is placed on the moment, the consumption of the music, the interaction between the performers and audience in the now.)

A great quote from Liz:

You think you’re hearing individual notes, you’re not. You’re hearing motion, you’re hearing what you expect to hear next. Your brain, listens for patterns, and then thinks in terms of expectations. As designers, I think that this is highly important, because we’re not thinking about individual pieces of information, yet we’re thinking about what is to come, what patterns can we imagine into the future.

That last point is the most important. Other than "looking different" the change doesn't do anything helpful. It's a huge usability loss for an awful lot of people.

— Scott Richie, “It’s time to fix window controls”

While design the UI for Weekly Retrospect I try to innovate but will only break convention when it serves a higher purpose than just being different.

The next time you’re constructing a user interface, you should absolutely follow Fitts’ law. It just makes sense. But don’t forget to follow the opposite of Fitts’ law, too — uncommon or dangerous UI items should be difficult to click on!

— Jeff Atwood, “The Opposite of Fitt’s Law”

It’s interesting that the examples he uses, I believe, are bad ones. I think the UI of GMail is worlds better than Wordpress, but that might be that the complete experience is so much better and not just about these specific examples.

52 Weeks of UX: 10 Principles of UX

3. Great User Experiences are Invisible: When people are having a great experience, they rarely notice the hard work that has been put into place to make it happen. This is as it should be...our job as UX professionals is to be so successful that nobody talks about us.

I completely disagree. A good user experience is noticeable and more importantly, memorable. That’s why I switched to Mac, that’s why I go to certain movie theatres even though there are closer ones, that’s why I always buy this one particular brand of car: that’s how you get loyalty.

Manufactured demand, bottled water.

This shows use that as consumers we are putty in the hands of designers. Product designers, graphic designers and even campaign designers.

Designers have the power to manipulate consumers into doing what they want.

This ties in with my illustration project at AUT where I have to design a cover for a TIME issue title “How Green is NZ?”

Doing What Comes Naturally

A lot of the secret sauce behind Natural User Interfaces is the understanding of the difference between abstract and direct manipulation of content. I’ve been talking about this a lot over the past couple of weeks and was delighted to see that Bill Buxton gave a half hour talk entitled “Doing What Comes Naturally” (Buxton starts at 2:13:40) at the end of MIX10’s second keynote.

Buxton started off with a very clear description of what Natural User Interfaces (NUIs), or what I’ve referred to as Natural Human Interfaces, are by using the example of a saxophone. He used a device modelled after a saxophone to simulate the sounds of a saxophone. This is a NUI, a direct manipulation. Compare this to a computer program where certain keys on a keyboard translate to the holes on the saxophone and you understand the difference between a NUI and an abstract UI. Buxton went further to say that what make the saxophone a NUI is not because it’s a representation of a real saxophone but that it’s a device that he already knows how to use because it respects the skills he’s already attained from learning how to play the sax.

He went on to demonstrate how this NUI could be used to simulate a flute and finally a guitar. Although different instruments, they all speak the same language and a note on one instrument translates to the same note on another instrument (albeit they are achieved through different interactions.) What made the saxophone a NUI was that Buxton didn’t need to learn how to use it.

Natural User Interfaces are interfaces that respect the skills acquired by users.

A guitar or flute would not have been a NUI for Buxton because he didn’t have the skills to use them, he could play a saxophone so a saxophone was.

If you found a PC 2000 years in the future and all trace of human kind had been lost. What would a good physical anthropologist infer about human beings today? And they’d say you have no legs, you have no mouth, you have an eye that can do fully saturated colours you have an ear that can do 200Hz bandwidth and you have one claw with about 80 fingers.

The gap between the tools we currently use to interface with computers and the way we interact with our world is clear. A computer is not built for US, we have been forced to learn how to speak the computer’s language. The most important part about technology is not the hardware, not the software it’s the wetware, the human being.

The important part of this equation is not what is the latest and greatest input technology. The important technology is to focus on is the Human Being, how we work, how we naturally interface. This is why I referred to these interfaces as Natural Human Interfaces, the Human part is important, it’s the most important. The input technology changes swiftly and constantly, but human beings change very slowly and so you will benefit so much more in the long term from understand human beings and designing for them and not a particular technology.

What is the NUI equivalent of WIMP? (windows, icons, menus, pointer) OCGM: objects, containers, gestures, manipulations.

Objects are the content in an interface. Containers make relationships between content explicit. Gestures are body motions that have meaning. They are discreet (you have to complete a gesture before it can be interpreted). Manipulations are direct and have continuous feedback. With gestures the feedback does not come until the gesture is complete.

— Luke Wroblewski, Notes from “Developing Natural User Interfaces”

Natural UI in use today.

I’ve been blowing a lot of smoke about this Natural User Interface (NUI) idea lately but that’s because I think it’s very important to the development of UI that is useable to anybody and, at the same time, a joy to use.

This is evident in two approaches to device design, Android/Windows Phone 7 Series Devices and the iPhone/iPad.

Hardware buttons are required for these devices (Android has 4 and Windows Phone 7 Series has 3) which is the equivalent of the mouse and keyboard on a personal computer (abstract interaction.) The iPhone doesn’t, instead it provides a viewport on which is displayed all the UI that you interact with using some form of direct manipulation.

To be clear, the iPhone isn’t a complete NUI experience. The iPad isn’t either, it still requires abstract concepts like clicking but the iPad is a much more complete NUI than the iPhone and the iPhone much more complete than most other devices.

On the iPad you open an album using the pinch gesture which “throws” the photos onto the screen. This gesture mimics the spreading out of the photos and the gesture’s result is what is expected when we spread out physical photos. On the iPhone the idea of photo albums is still very abstract, a collection of files in “folders” displayed as a list.

On the iPhone, a strike-through gesture (known as “swipe to delete”) is used to delete items from a list. This action mimics the crossing out of an item from a physical list and the result is a request for confirmation of deletion. Other devices require you to use arrow keys (or touch) to select an item, then use other buttons (be they hardware or virtual) to display a menu from which we select the “delete” command.

The contrast between these simple, natural, gestures and the abstract actions required when using hardware to abstract interaction (be they a “Menu” button on a phone or a mouse pointer on a personal computer) show the extent to which Natural User Interface improves user experience and reduces the need to learn how to abstract reality into computer concepts which means we can easily pick up these devices and already know how to use them.

Natural User Interface seems to be a technology that both Microsoft and Apple are heavily investing in. Microsoft Research has done a lot of public work in this arena and in terms of consumer products coming out we’re seeing Project Natal from Microsoft and the iPhone/iPad from Apple. The future of User Interaction Design is an exciting one!

The Anatomy of Scrolling

Developing Weekly Retrospect’s UI has got me thinking about a lot of different interactions that I’ve never thought about before, specifically because of the difference between abstract manipulation of content (using a mouse) and direct manipulation of content (using your hand.)

I never quite understood why my grandparents said “Go up” when I used to scroll email or web pages for them (before she learnt how to use a mouse) when she actually wanted me to scroll down. Even my mother’s older sister would also use the terminology of “scrolling up” when she actually meant scrolling down.

I now see that they saw the content moving upwards so they thought it was “going up” when in fact I was moving the “viewport” down. The viewport is what we see at any point in time. When we read a book we move our viewport by looking over to the next page. When we read a newspaper we move the viewport around the the newspaper that we hold in front of us. Very seldom do we see the content moving around within our viewport (unless it is actually moving), instead it is our viewport that moves around the content. So when they see content moving up within their viewport they interpret this not as the viewport itself moving down the page but as the physically content moving upward.

I came across this issue when coding the Natural Human Interface interactions for Weekly Retrospect. Keyboard keys move the viewport around (like traditional scrollers) so a right arrow key will move the “viewport” to the right. To achieve the same effect with a Natural Human Interface we have to come up with an interaction (or gesture) whose effect in reality would mimic what is being shown on screen. As what is been shown on screen looks like content moving from the right of the viewport to the left of the viewport, an appropriate gesture is one which would physically move the content, like dragging from right to left.

So a motion which goes toward the left translates to a keyboard key that points to the right. This isn’t amazing or revolutionary, just something I thought was interesting. An understanding of abstract (Mouse and Keyboard) and direct (Natural Human Interfaces) manipulation and why they relate to each other the way they do.

This is also why I believe the iPad is going to be so revolutionary. My grandparents and aunt who never got the hang of abstract manipulation won’t need to because their interaction with the device will be direct (natural).

Swipe to Interface

Bill Gates has been going blowing the horn of Natural Interfaces for a long time and for as long I thought it was along the lines of his tablet computing spiel, however both seem to be coming to fruition at the moment. Bill Gates’ longtime friend, Steve Jobs, looks like he will finally be able to make a success of the tablet form factor and Bill’s other Steve, Balmer, is leading his former company to delivering Project Natal this year, a project that will deliver on the promise of Natural Interfaces.

Both these products require a new way of thinking about interaction design. Both are much clumsier than the mouse or keyboard and so interaction needs to be simplified. Since the resolution of control has decreased dramatically in these form factors, the resolution of the objects that need to be controlled must be scaled up.

Soon websites will be available not only on your computer screen but also on your iPad, your iPhone and your television screen hooked up to Project Natal or similar. The old way of interaction with the web won’t work with Project Natal and although possible with the iPad/iPhone it definitely isn’t ideal. I wanted to design a UI that would work across all these different platforms, and work well.

I recently launched the Weekly Retrospect which was originally going to be my attempt at curating and designing an online exhibition of the things I found (and most probably posted to this website) that week. What I found instead was an opportunity to rethink the way users interact with websites.

This will be an ongoing experiment as I try to refine the interface for various modes of interaction. Currently the website works without Javascript (or CSS), using the keyboard or mouse. It works well on portrait and landscape screens (your laptop and iPad catered for.) The next step is getting this to work with some sort of Human Interface — translating the swipe of a hand into the next arrow on a keyboard.

I found iGesture (pictured above) but haven’t yet figured out how to get it to work. I will continue to try and work on this and see how I can develop this style of UI.

Currently navigating between items and issues is simple enough, navigation within an item (say playing the video or jumping between links in an article) is not quite yet possible but I hope to introduce this along the way — most probably making use of the Enter/Return key to play video/launch full-screen image/launch the permalink/reveal interface to jump between links in an article.

Natural Human Interface, meet the web.

So I’ve posted about my experiments with UI lately, specifically trying to bridge the gap between point-n-click, touch-based and Natural Human Interface (ie. Gestures, at the moment with my hand in the air, a la Project Natal.) Here’s a demo video of Weekly Retrospect working with iPinchMe, a Mac app that interprets gestures out of video received from your iSight feed

I’d like to thank the developer of iPinchMe, Andreas Pfadler, for, firstly, developing this app, and secondly, helping me write the script to get it to work. After some clarification from Andreas, this is the script that you use:

on handle_event(event_msg, state)

	if event_msg is equal to "right" then
		tell application "System Events" to key code 124
	end if

	if event_msg is equal to "left" then
		tell application "System Events" to key code 123
	end if

	if event_msg is equal to "up" then
		tell application "System Events" to key code 126
	end if

	if event_msg is equal to "down" then
		tell application "System Events" to key code 125
	end if

	if event_msg is equal to "green" then
		tell application "System Events" to key code 36
	end if
	
	if event_msg is equal to "blue" then
		tell application "System Events" to key code 36
	end if


end handle_event

After you’ve initialised iPinchMe, make sure Safari is active so it is sending the commands to the right window.

Online representation is the same as physical representation: what your clothes, friends, vehicles say about you.

— Dr. Urs Gasser in “What Companies Should Know About Digital Natives”

The man who coined the term “Digital Native” explains exactly who a Digital Native is. The above quote identifies what identity means to these people.

I view social media as a way to shape my public image. I only post my best photos on Flickr and only update Twitter or Facebook with the messages I want people to read. Having an always-on, always-updating social media mindset muddies the water too much for my taste.

— Stephen Hackett on Robert Scoble’s comments about automatised social media.

So today I strode onto the shampoo aisle and was confronted by a wall of shampoo, at which point I was lost. Totally. I'm standing there and I have no idea what shampoo I'm supposed to buy. Ours is in a big white bottle, but there are many big white bottles here.

— Andy Rutledge, “Buying Shampoo”

It’s sad when an entire product category say the same thing to the point at which the product becomes commoditized. This does create a gaping opportunity for creating a compelling brand (like Moleskin and Field Notes have done.)

We are doing an exercise at University where we switch the voice of two different products, I’m considering Raro/Malibu or Fanta Grape/Wine to see how shocking the effect is.

Concepts we live by

How people understand and often misunderstand each other fascinates me. How is something that makes so much sense to me so completely misunderstood by someone else. Why is it so hard for someone to learn a new language, despite knowing the words, and even speak their own language in a different country.

When I go back to visit South Africa and speak Afrikaans to the people there, despite the fact that we are both speaking the same language, the way we talk is completely different and what I say is not understood by them and what they say confounds me as to what they mean.

Lakoff, G. & Johnson, M. argue in “Concepts we live by”, Chapter 1 in “Metaphors we live by” that this is because our thought processes and understanding are largely metaphorical, metaphor is nor just a matter of language, that is, of mere words. We shall argue that, on the contrary, human thought processes are largely metaphorical. (Emphasis theirs.)

They use the example of the extended metaphor that our culture uses, Argument is War.

• Your claims are indefensible.
• He attacked every weak point in my argument
• His criticisms were right on target
• I demolished his argument
• I’ve never won an argument with him.
• You disagree? Okay, shoot!
• If you use that strategy, he’ll wipe you out.
• He shot down all of my arguments.

Without the understanding of the extended metaphor, our intentions are misunderstood. It’s not just about understand the language, the words, it’s also about understanding the society, their thinking, the extended metaphors they share.

In terms of design, this is also applicable. Just as we speak based on metaphor, we also design based on metaphor. Someone won’t understand your designs if they don’t share the same extended metaphors you have, so when you design for an audience it’s important to understand that market and how they think. Don’t be surprised if they don’t understand your work if you didn’t take the time to learn to understand them.

East vs. West

A continuation of the journey of exploration of culture and understanding of humanity that I’ve been on recently.

If you understand the difference between ‘the world’ and ‘my world’ you understand the difference between logos and mythos. ‘The world’ is objective, logical, universal, factual, scientific. ‘My world’ is subjective. It’s emotional. It’s personal. It’s perceptions, thoughts, feelings, dreams. It is the belief system that we carry. It’s the myth that we live in.

Culture is a reaction to nature, and this understanding of our ancestors is transmitted generation from generation in the form of stories, symbols and rituals, which are always indifferent to rationality. And so, when you study it, you realize that different people of the world have a different understanding of the world. Different people see things differently: different viewpoints.

There is my world and there is your world, and my world is always better than your world, because my world, you see, is rational and yours is superstition, yours is faith, yours is illogical. This is the root of the clash of civilizations.

What is good without bad.

A recent uproar about the introduction of internet filtering in NZ follows on the heels of the great outrage at the Australian government’s choice to enforce internet filtering. In NZ, the filtering is optional and so far Telecom, TelstraClear and Vodafone have all opted in.

EDIT: Orcon is also working to implement this. And MaxNet CEO, John Hanna, says "Filtering out child pornography is also very much in line with our company values — our customers would be disappointed to hear if we weren't participating. So participation for us has always been a no-brainer." (source) Ask your customers if they want their internet filtered, not whether they think child pornography should be banned. - LW.

The reason we require internet filtering is because of the prevalence of child pornography in little old God’s Own apparently. Who can oppose this move without seemingly supporting child pornography? I strongly oppose child pornography almost as much as I oppose this internet filter. Child pornography is an evil that needs to be eradicated, sure, but it affects a handful of people and may never have an impact on most people’s lives. Do not read this incorrectly and think that I suggest child pornography is not a big problem and not one that seriously harms those involved because that is not what I’m suggesting.

Child pornography destroys lives. Internet filtering, and the bigger issue, a suppression of freedom and choice, destroys our society.

We all know and have read the dystopian novel by George Orwell, “1984” and we all know that one of the slogans of the Party is “Freedom is Slavery.” The idea that by restricting our choices to only good ones, removing all evil alternative choices, will lead to a better society. This idea is flawed in that it doesn’t understand that good can only exist where evil is an alternative.

Who is a good citizen if they never turn from a bad choice? What makes a good decision good, if there is no choice involved?

Freedom is the foundation of modern society. Yes, some people will abuse that freedom, but most do not. It is our choice what we do, what we see, what we hear and what we say and not that of our government. It is our choice to be good and not to be evil, for if the government take away that choice, then nothing can be evil, or good.

Today filtering is used to eradicate something we all agree is evil, but tomorow it may be used to eradicate something a little more grey, between obviously good and obviously evil. And the next?

Yes, freedom means some people will make the wrong choices, but it protects us from ignorance and allows us to choose to be good.

Nobody (save the other craftsmen) will appreciate half your skill. But you may spend endless years of happy experiment in devising that crystalline goblet which is worthy to hold the vintage of the human mind.

— Beatrice Ward, “The Crystal Goblet”, Excerpt from a Lecture to the British Typographers' Guild

Our work is to make someone else’s work shine. Whether we are typesetting an author’s novel, designing for a bands album cover, coming up with a billboard or a website for a company’s product. Our work, is to make our client’s work, stand out and be clear and understandable.

Modernist design ethics suggest that design should be invisible, form follows function. This view point on design is debatable, my favourite retort has been along the lines of “no one wants to drive an invisible Ferrari.” So what is design’s role? To disappear or to awe?

Cameron Moll's Colosseo

Beautiful type, beautiful press. Educational and inspirational video.

Dan Cederholm's Bit Monsters

Another web designer flirts with the press. With the introduction of the iPad most people announced the death of print but lately those whose work is almost entirely digital have found in print something that cannot be found in digital design, tangibility.

Beautiful redesigns of these classic book covers (with process sketches and commentary!) Reminds me somewhat of the re-imagined covers in the Classics apps.

(via Matthew Buchanan)

Of the books we do print — the books we make — they need rigor. They need to be books where the object is embraced as a canvas by designer, publisher and writer. This is the only way these books as physical objects will carry any meaning moving forward.

— via Craig Mod, “Books in the Age of the iPad”

I think we need to change our expectations of a book. We now publish experiences (or apps, if you will) instead of books. Why have a textbook when you can have an app provides the same information through a better experience?

What happens with novels though? And other text-reliant experiences where the value is in the form just as much as the content.

iPad Application Design: back to reality.

I thought the success of the computer was because it was abstracting reality which enabled it to speed up workflow. Apparently the future is skinning that abstraction so it simulates reality.

My grandmother will understand it much easier I agree.

Brilliant game design from Shaun Inman. The level of “power-up” is not indicated by the size of the character or in fact anything to do with the character itself but instead affects the game-world. A higher level of “power-up” produces a higher bit world. 2bit, 4bit, 8bit and finally 16bit power-ups.

This is something I can’t wait for!

Distribution of colours of the flags of world (individual country pie charts through the link.)

Photography by Burak Arikan that shows the importance of simplicity and uniqueness in flag design for recognition purposes. I assume the same can be applied to most logos with similar success.

Contemporary reaction to Jesus must have been similar. This man is crazy as hell. The difference I see is that Jesus gave and this man takes.

Not until we are lost do we begin to understand ourselves.

— Henry David Thoreau (via o-myriad-o)