posted Jan 12th 2009, 10:46 under haptic, phd, research
I'm currently doing my PhD at the Umeå Institute of Design with the continuation of my HAPI project. I will dig deeper and explore thoroughly the domain of haptic interfaces and techniques in interaction to get there. My PhD topic is Towards Mobile Haptic Interfaces, sketching Multimodal Interaction Design.
I started a new online workbook/blog for my PhD activities and thoughts. Is is available at http://www.partly-cloudy.com
posted May 15th 2007, 00:07
final presentation [pdf, 11.9 MB]
video files not included
preliminary report new version coming soon
posted May 13th 2007, 08:35
Say farewell to Touching People. My degree project now has a new name:
HAPI: haptic interaction for mobile devices
The initial name was interesting (in relation to Nokia's Connecting People), but it could also be associated to various meanings that didn't really related to my project. So hence the new HAPI name.
I registered the domain www.hapi-project.com so both addresses now direct to the same site.
posted Apr 27th 2007, 22:50 under accelerometer, docomo
Some of the new FOMA 904i phones now include accelerometers. I couldn't find extensive details on how motion sensing is used for real applications (beside launching apps and playing games). But it's interesting to note that such sensors are finding their way into more phones nowadays. The Nokia 5500 Sport also has 3D accelerometers embedded.
via www.intomobile.com
posted Apr 23rd 2007, 09:36 under bluetooth, processing, WHC
This eleventh WHC is a direct refinement of WHC 9 where I was using the 6DOF unit to navigate a simulated menu on the laptop. Using the laptop is very convenient for developing my code. The processing power is ample and I can use various libraries for UI and effects. One thing I realized while testing with the portable computer as a screen is that people don't look at the device while manipulating it. Much like playing with the Nintendo Wii. It's quite normal, you want to see what is going on as you move the portable device. It results in quite large movements or gesture, not really want I intended (and how sensors/parameters are set in the code).
My goal for this WHC was to put the visual feedback (screen) back into the users' hands. I had many options to accomplish this, but I went for the fastest and simplest one: use a real phone in my mock-up. Actually it's not the simplest one, as there are multiple layers to get this working. But the end result is there, things append on the phone as you manipulate it. And all wireless !
The 6DOF unit + battery fitted with the N73
... more
posted Apr 22nd 2007, 01:06 under arduino, processing, vibration, WHC
After the quick success of the vibrating duet (WHC 7) it is now time to try out a full 5 nodes version. Actually this project is a direct revisit of one of my first mock-up, the penta-vibrating grip (WHC 2).
Much of this has already been done before, it is more a matter of redoing and repacking the various parts. One nice thing is that my sequencer application works very nicely with this grip (replacing the 5 pistons unit). The Arduino code stays the same also, only the actuator/vibrator relays need to change. For the records, we are now using the regular 4420 IC instead of the 4421 ones. The pistons required inverse signal (no input - output on), where the vibrator motors need direct (no input - output off).
The grip drilled to receive the small vibrators.
... more
posted Apr 20th 2007, 22:58 under feedback, keys, phones, tactile
How to get tactile feedback through a continuous surface? Is it possible to get good and rich tactile cues without using discrete and physically isolated buttons? To my surprise it seems like it. More and more handset manufacturers are indeed using 'skinned' buttons and keys on their devices, where an outer layer wraps/embeds/hides the physical keys. ... more
posted Apr 19th 2007, 00:36 under gesture, processing, WHC
After playing a bit with the 3-axis accelerometer (WHC 1 and 3), it's now time to get serious with IMUs (Inertial Measurement Units). I am now using the 6DOF v.3 unit from Sparkfun (link). It offers 3 axis-accelerometers (X, Y, Z), 3 gyros (roll, yaw, pitch) and compass headings (X, Y), all packaged with a Bluetooth link. Quite a beast! Check this wikipedia entry to know more about IMUs and how they work.
The goal of this week is to implement some sort of menu navigation (moving along one axis) using small gestures/movements. The tilt-slide interface has already been developed and published before. I am going more for quick hand movement that would allow you to move up (or down) 1 unit, 3 units or go all the way up. This calls for some filtering and peak detection algorithms.
My code in Processing is far from perfect but it fulfills my needs for now. Remember, the goal is not to come up with a production class software. It is more a hardware sketch to be able to test some ideas quickly.
The base is there and hopefully the application and algorithms will be refined to be more precise and recognize more gestures in the coming days and weeks.
The marvelous 6DOF unit from Sparkfun.
... more
posted Apr 16th 2007, 22:18 under nokia
What a lovely monday morning. A big box was waiting for me on my desk as I stepped into the studio this morning. Surprise, two Nokia N73 landed in Umeå! Two because my colleague Kristofer also has his degree project supervised by the same tutor as me. So many thanks to Mike Kruzeniski at Nokia for the phones. They are superb. It is bigger than my K750i, but the screen and control keys are much nicer. And it runs the latest Symbian OS with great Java support and Flashlite 2.1. I will definitely put this phone to good use for my degree project. Check back soon for some quick and dirty prototyping with the N73!
posted Apr 11th 2007, 21:52 under arduino, processing, WHC
In this challenge, I decided to migrate the actuators and sensors into some sort of portable devices. I felt that that putting your palm on a desktop device is quite different than holding something in you hand. The results are two matching portable devices: one controller with the light sensors, and one poking device with the electro-mechanical actuators. They now both have 5 nodes and can be remote controlled by the computer running an improved version of the Processing sketch.
Note to me: serial communication is problematic. Incoming buffer data is not handled properly resulting in some sort of inconsistency over time. [fixed now !]
Building the new handheld receptacle for the actuators.
... more
posted Apr 9th 2007, 21:06 under tactile, vibration, WHC
This week's new Hardware Challenge is very simple. It took only about an hour or so to build. It consists of a cardboard box with two vibrating motors. It is a revisit of the failed penta-vibrating grip (WHC 2). This time, I tried to isolate the vibrating units from each others and minimize their mechanical link to the box/supporting structure. The only thing that keep them together is a thin layer of foam on glued on top.
You know what, it works... more
posted Apr 8th 2007, 12:05 under haptics, research
Getting in Touch: Virtual Maps for the Blind from Scientific American.
An article on haptic navigation system developed by a group of researchers at Aristotle University of Thessaloníki in Greece.
zdnet.com also presents the article and adds some notes/references to it:
http://blogs.zdnet.com/emergingtech/?p=535
posted Apr 1st 2007, 16:17 under controller, multitouch
This is not totally in line with my degree project but I found it while browsing Processing resources (via Flight 404). So Lemur from the French company JazzMutant is a multitouch controller for live performance. I won't mentioned more since their website is doing a good job of presenting and explaining the device.
As an intro you can watch this video
posted Apr 1st 2007, 15:38 under interaction, one-handed, research
Mobile phones today usually support one-handed interaction. You can use two hands to speed up some tasks but most operations can be completed using only one finger or hand. PDAs on the other hand, usually involve two-handed interaction with their touchscreen and point-click actions. Generally one hand is used to hold the device as the other one act as a controller (actuating, moving and pushing buttons).
It is a sure bet that we will see a lot more touch screens appearing in mobile phones over the next few years (iPhone and such). Sure they have some advantages (customizable interface) but also present some challenges. I won't go into details here but I just want to mentioned this nice research done by the HCI group at the University of Maryland. Their work was presented at CHI 2005 and it involves developing interfaces to support one-handed use of small devices.
http://www.cs.umd.edu/hcil/mobile
posted Apr 1st 2007, 11:53 under haptics, immersion, technology
Immersion is probably the biggest name in the haptic technologies market. They have been around since 1993 and hold +600 patents for various haptic systems. Most of their products and applications are targeted at businesses and large corporations (due to the high cost), but new offerings are finding their way into mainstream market.
Their range of products is quite impressive: from automotive systems to medical and gaming applications. Mobile devices are also under their portfolio with the VibeTonz system. At first sight, it looks like personalized tactile ring-tones, but I guess the possibilities are much broader than this. I never encountered such device or demo unit, but I heard some report that it is quite convincing and believable.
If you don't mind cheesy music and an over-excited host, check this 9 min. Flash video from Tech Closeup. It showcases some of the Immersion products and technologies.
The Immersion website also offers good information on haptics:
posted Mar 31st 2007, 23:27 under arduino, processing, WHC
Over the last few days, I have been refining the poking machine with various components. First I completed an analog controller to trigger the actuators. It is composed of simple photo-resistors that are sampled by the Arduino board. Getting my hands on more photo-resistors (I had only two in my stock) turned out to be a real challenge. It took me more than four hours of intense research to browse and find a store that would sell and ship them to Sweden. They finally were delivered pretty fast (next day) from Elfa with a custom order number. The idea of the board is to act as input device to control and record sequences for the array of actuators.... more
posted Mar 31st 2007, 11:47 under experience, sensorial
Last week, a colleague of mine suggested that we watch The Kentucky Fried Movie. It is an old comedy from 1977. Some parts are quite funny, others aren't. There is a scene where one character goes to the cinema where the presention is offered in "Feel Around". I guess this is the ultimate sensorial user experience!
Get Flash to see the movie
posted Mar 26th 2007, 01:45 under arduino, tactile, WHC
My first attempt at building an array of stimulator was not very successful. Actually, I would qualify it a total failure. But hey, life is not perfect and we learn from our mistake. So this week's WHC is my second try at builind an array of stimulators to encode information to the use... more
posted Mar 22nd 2007, 01:39 under 770, nokia, test
Thanks to Mike Kruzeniski, my tutor from Nokia, I got my hands on a nice mobile device yesterday. The Nokia 770 is an Internet Tablet with a gorgeous screen (800 x 480). This is the first time I get my hands on such a device. I have never used PDAs or smartphones (that may be similar) but I have to say that I am pleasantly surprise by the quality and the in-hand experience of the 770. This type of large mobile device is quite enjoyable to use initially (minus the stylus thing). The graphics are very nice also, web pages and images/videos alike. I wouldn't personally buy such a device (too expensive for my needs), but congratulations to Nokia for this well executed combo of hardware and software.
Sure the device is not perfect, it has some issues and limitiations. A new model (N800) was launched last January as a followup to the 770, mainly adding a web-cam and a faster processor. In the coming weeks, I hope to use the 770 to test various things regarding my project: two handed interaction, large device manipulation, flashlite interface mockups, etc.
Great device with a large screen.
... more
posted Mar 21st 2007, 23:07 under design, mobile, webcast
from the very good Small Surfaces blog
Jared Benson’s mobile design webcast has been made available by Jupiter. The 50 minute webcast goes over some of the basic things to think about when designing for mobile phones.
link to the webcast
posted Mar 15th 2007, 23:40 under arduino, WHC
This fourth Weekly Hardware Challenge consists of getting a LCD screen to work. The unit I have is a Color LCD 128x128 pixels Nokia Knock-Off (from sparkfun). Sample code is available for many different microcontrollers and getting it to run with a Basic Stamp was a breeze. Unfortunately, no code was directly available for Arduino boards (the one I want to use for this project). With a little bit of digging on online boards and forums, I discovered that one user got it to work without much problem. I sent him a quick note and he generously sent me his source code (also now available here). Great, big thanks Amp! That is the beauty of online communities.
So after some basic rewiring and configuration for the Arduino board, I got the LCD to display test content. Hourray! The main consideration is to drop the signal level from 5V (Arduino) to 3V (LCD). A simple resistor voltage divider did the job. It might not be the most compact setup as of now but it's working.
update 16.03.2007
fully wireless version with Bluetooth Arduino and battery pack. (see last im... more
posted Mar 15th 2007, 01:09 under haptics, mcgill, nokia, publication, touch
The Economist just published a nice article about haptic technology and how it can help our interaction with digital devices. A great primer on the topic. I find it quite amusing they cite Prof. Hayward from McGill and Nokia in the article. It's not very far from my project :-)
How_Touching_The_Economist_2007_03_10.pdf
posted Mar 15th 2007, 00:12 under presentation, video
Halfway through already! Time flies, really. So here are my mid presentation files. First you will find a 5 minutes video intended to summarize and communicate our project to Paula Guntaur, our external examiner. I have to say it was quite a challenge to keep it short, my first edit was 9 minutes long and I felt I was skipping on a lot of stuff. Anyways, it may not communicate my project properly, but I think we get the general idea.
The second file is a PDF of my one hour presentation/discussion with professors and tutors at UID. There were many video clips embedded in the presentation, and unfortunately the web medium is not so great at supporting these (yet). So just plain PDF file for now. Also, you will notice that a lot of slides don't make sense on their own (without me talking about them). I prefer to build slides that complement my live discussion, in opposition to have everything written on the screen. I think it makes the presentation more enjoyable and lively, but it's not so good for post-presentation and reviewing activities. Oh well, we can't have it all (and I'm still learning).
Thanks to all for your comments and good feedback. It will definitely help me move forward with this project. 60 days to go.... yikes!
posted Mar 12th 2007, 11:05 under language, multitouch, switzerland
I recently discovered this diploma project tangent from two students out of Zurich, Switzerland. Even though this project deals with tabletop screen interaction (not mobile interaction), I really like their approach and interaction techniques language.
When digital objects can be manipulated directly by hand, the interaction principles of conventional mouse-keyboard-interfaces are not valid any longer. In order to be able to develop applications for sensitive surfaces, it’s therefore necessary to first create adequate interaction principles for these novel computer interfaces. Tangent enables us to transfer interactions with physical objects known from everyday life to the digital medium.
... more
posted Feb 28th 2007, 23:23 under tap, WHC
I was really impressed by the demo mockups from Jukka Linjama (Nokia) at the TEI conference (see video below). Understanding the new tap interface was instantaneous using the tweaked phones. Now back in Umea, I wish I could have such demo devices to show the concept to my colleagues and users. Remember: touching is believing!
The tap system is based on accelerometer sensors and is quite easy to implement actually. I decided to give it a try at building a version of my own with the components I have on hand. I only dedicated 2-3 hours to develop the system but it's wo... more
posted Feb 16th 2007, 02:23 under baton rouge, conference, research, TEI
I am now in Louisiana for TEI '07, the first international conference dedicated to research in tangible and embedded interaction. The program looks very exciting and I am looking forward to meet and discuss with other participants at this event. Yes there are big names presenting their work (Ullmer Brygg, Tom Rodden, Ivan Poupyrev), but the overall venue seems very diverse and international.... more
posted Feb 14th 2007, 15:07 under haptics, montreal, research
One word to describe what I saw today at McGill's Haptic Lab: wow! I have to say that I am really impressed with the work I discovered today from Professor Vincent Hayward and Ph.D. Student Vincent Levesque.
text to be completed very soon...
posted Feb 9th 2007, 23:37 under conference, geneva, LIFT
I am now in Geneva, attending LIFT: a conference about the challenges and opportunities of technology in our society.
This conference is at its second edition and is boosting +500 participants from diverse fields such as web development, blogging, design, and everywhere between art and technology. It is quite a heteroclite audience, but the common denominator seems to be technology, especially people with laptops. Welcome to connected-land, where people type, browse, click and blog more than they eat!
More seriously, the 3 days of the conference were packed with talks and activities. Some of them were good, some of them were very 'soft' I would say. I expected more thought-provoking or eyes-opener presentations, but I am a bit new to these conferences. Maybe I expected too much. Nevertheless, I really enjoyed most of the activities. Also, it was a good forum to meet people, great thinkers or just plain good colleagues in this large technology+people communi... more
posted Feb 7th 2007, 03:22 under discussion, IDL, research
Today I was able to meet with Tomer Shalit, an entrepeneur and researcher here in Umeå. Tomer has been involved with haptics and tactility for many years, working on various projects at IDL and setting up ReachIn.... more
posted Feb 1st 2007, 23:09 under arduino, tactile, vibration, WHC
For this second Weekly Hardware Challenge (WHM), I decided to explore tactile signal resolution: is it possible to easily identify the source of a stimulus? Could you have a tactile display, composed of 'haptic pixels' that can convey information adequately? I read a good number of papers about this topic and, I thought it would be interesting to try it out even if my hardware resources and capabilities are very limited.... more
posted Jan 30th 2007, 11:25 under gesture, IMUs, motion
Today I found this new company called InvenSense that focuses on developing sensing products for mobile and consumer applications. It is very interesting to see a company dedicated in pushing motion sensing into mobile devices. They say (naturally) that this technology is the next big thing in mobile devices, and in particular they see it enabling
... smarter user interfaces using hand motions and gestures as commands for more natural interaction.
They don't show actual concepts or prototypes for now, but it sure is in line with some ideas I am exploring during my degree project.
InvenSense's website offers good whitepapers and information on the topic. Check out their libray page, the video titled Gyroscopes and Accelerometers Compared is a nice overview on motion sensors and IMUs.
posted Jan 29th 2007, 23:11 under future, nokia, research
I recently discovered that Nokia, my sponsor for this project, is publishing a lot of interesting material on the web. It ranges from future concept videos to ethnographic studies from around the world. Here are some of the documents and materials I found interesting.
Nokia World
www.nokia.com/nmc
The annual gathering for Nokia and its collaborators (operators, partners, etc) in the mobile community. The dedicated website offers webcasts and presentation files... more
posted Jan 29th 2007, 00:44 under calendar
I published the timeline section minutes ago. It is lacking some functionnalities but I think it's better to have it now in static format at least.
posted Jan 27th 2007, 00:42 under bluetooth, processing, WHC
For this project, I decided that a weekly hardware challenge (WHC) would be a good way to start prototyping early and keep motivated about it. It doesn't have to be complicated, but do a least one hardware step, evolution or implementation per week. I don't how good I will be able to stick to this plan, but I'll try my best.
Earlier this month, I ordered a lot of new electronic devices and sensors. Some of them arrived promptly in Sweden, others are still in transit unfortunately. For this first instalment of the WHC, I spent a bit of time to check out some of the little gems I received so far and decided to go with the WiTilt from Sparkfun. This item is a triple axis accelerometer combined with a ... more
posted Jan 26th 2007, 11:58 under haptics, mcgill, prototype, research, tactile simulation
I recently discovered that very good research on haptics and tactile simulation is conducted in my hometown (Montreal, Canada). McGill University's Haptics Lab has been studying the touch sense and building really interesting prototypes over the last few years. Some of their newer devices and projects are directly in line with my degree project. Their THMB device looks particularly interesting: they are investigating the use of tactile simulation and feedback in a modile device.
I contacted them yesterday and I am very happy to report that I will able to visit the Lab at McGill next month. I can't wait to see their work in person and hopefully have a chat with the researchers. Touching is believing!
Official Lab's homepage: http://www.cim.mcgill.ca/~haptic
Sub-site with latest projects: http://laterotactile.com
posted Jan 24th 2007, 14:34
It took a bit more time than expected but here it is. It might look a bit empty now but a lot more content will be added very soon.
The website in itself is a work in progress. After testing different ready-made web packages (like Wordpress and Drupal) and being not so happy about them, I decided to build my own custom site with CakePHP, a php framework that was released earlier in 2006. All this means that I have a bit more learning and initial coding to do in order to have basic functionnalities (add, delete, edit content) online. It represents more hours of web development, but it gives me a good excuse to refine my php skills and explore new topics (version control, MVC, etc).
So please bear with me for the first weeks or so, everything won't be perfect from the beginning. My next major challenge is coming up with a decent timeline or calendar module. I tried the route of integrating Google Calendar API's in the framework but without success so far. I'll see what I can do...
