Article: “Tangible Interaction”

The current issue of interactions magazine features a cover story called Tangible Interaction = Form + Computing, which the authors, Mark Baskinger and Mark D. Gross, describe as a combination of physical design and computing. They reference NYU’s Tisch School of the Arts (ITP) as one of the first schools in the world to have embraced tangible interaction design. In their article, they speak of the switch as the simplest of sensors, and reference Pd (a software program specifically for sound design), Arduino (for gadgetry) and Lilypad (for wearables)…all of these part of our non-core curriculum.

As an ITP student, one of the most heartening parts of this article is the section on “Where Do Tangible Interaction Designers Fit?”. They say, “The field is still wide open, but one thing is clear: We’re likely to see more, not less, programming in things, and a lot more experimentation.” It’s nice to hear that there’s still so much more development to be had, since it means I’ll eventually have a job.

In reference to ITP, well, we have a lot of artists who come through and I don’t know how many of them would really think of themselves as a “tangible interaction designer” even if that’s really what they are, if you have to call it something. And, I do think the article lacks a bit in the artistic and simply wonderment aspects of what we do; not everything is designed for more than just enjoyment of life or experimentation. Not that they need to, but as they say they’re modeling their studies at Carnegie Mellon on programs like ours. So, maybe, there’s a PhD in my future…I did after all quote Dr. Gross in my ITP admissions essay.

Liveable Interactive Music Player


My final project for Physical Computing eventually ended up to be an MP3 player that encourages you to take care of your houseplant. This evolved from a previous idea of making a plant that is more like a pet than a plant. I changed my mind after getting some good feedback from my class when I presented my idea. One of their suggestions was to focus on the emotional aspect of how the plant makes you feel. They also advised that care of the plant needs to be an important aspect of the project.

This is how it works: Press the moisture test button. If the soil is dry, the plant will cry; otherwise, it will laugh. If the soil isn’t dry, you can use the MP3 player. You can advance forward and back through an array of music files. You can also pause the music.

Test Button
Test Button

Not only do you care for your plant, it seems like your plant cares for you, too. Although the interface for this project ended up being dainty and pretty, I do admit that 90% of my time on this project seemed to really be spent on code and trying to get things to work. I’ve written about this project in previous posts, so I won’t recap what I’ve discussed in those. But, they’re here if you are interested in reading through them and understanding more about how the project was generated.

Process Description

Because I wanted to embed the interactive mechanics inside the plant, I spent about 1 week on the wave shield and, while I did get it to work, in the end still couldn’t get it to work so well. Some of the issues I had with it were that I didn’t understand the documentation clearly, the code is in some C-language, and my SD card wasn’t formatted correctly. Eventually, I did get it to work, but by that time Minim was looking pretty good. So, sound was now going to be handled by Processing. I also tested out different motion detection sensors to see what types of results I could get that would fit my needs best. Some of the issues I had were that the sensors were too bulky to elegantly embed inside the plant or the serial data I got from the Arduino was too sensitive.

Meanwhile, I was also thinking about my ICM final and how I wanted to create something separate from my Phys Comp final. Originally, I wanted to make a radio that streamed internet radio stations. However, when I discussed this idea with more knowledgeable people and looked it up online, I got the impression that it would be very difficult to implement. So, I left that idea behind and went to my next idea, which was simply an MP3 player. Wanting to make something more interactive and pretty, I started looking up options for Processing and Minim, and what type of visual display I could make. I looked around for an “audio visualizer” on and found this sketch to work with. After that, I spent a little bit of time everyday trying to figure out how the code worked and augmenting it bit by bit to create something new.

By now, it became obvious that I should just put the two projects together. Deciding to use Minim for my Physical Computing project was a huge relief. Not only was it easier to use and had documentation I could understand, it was also easier to get help since Minim is something that other people at ITP have actually worked with on a somewhat frequent basis. (It’s also in the book).

Plus, I could research more about Minim while I was away at a conference in London for a few days. When I got back the States, I went back to the prototype I’d set up before I left using FSRs as triggers for the playback buttons. The FSRs I switched out for exposed wires, because I really didn’t need the analog input values the FSRs provided. (Plus, I seem to have a tendency to destroy my FSRs.) What I really needed were simple HIGH and LOW values from a digital switch which I made out of small sheets of copper.

To test the soil for its resistance to electrical current, I made what could easily be called the simplest switch ever. Add 2 nails + 5V + ground to wet soil. Test for current. Actually, I learned how to do this from reading the internet, the blog of another student,and talking to yet another knowledgeable person.

As I understand, this switch works when the resistance of the soil changes enough to allow a current to flow more freely. At first, I tried this out with just a bowl of water, but I think maybe it shorted out (?) – you really need a plant or at least the soil to get this to work properly. I also added an LED for my own visual feedback.

So, now I had the buttons and the plant. Eventually, I also had a pretty nifty audio visualizer (which at the time of this writing, I may still try to improve upon). So, finally, I just had to assemble the pieces.

Pretty much I just found whatever I had lying around. I found a picture frame in the junk shelf and used that for the frame. I also had some pieces of cork lying around for a bulletin board I never put on my wall, and I used that to cover the frame and to be a backing for the buttons. The whole construction phase literally took just 1 day. Most of my issues with this project were with the code. Luckily there are more knowledgeable people around who can offer help with the code. It also made it a lot easier to test the switches when the wires were not stuck in a plant.

Now that I’m done with the simplest of functionality, to improve it here are a few things I could do:

  • Make a sturdier interface – use something other than cork
  • Label the buttons
  • Provide more feedback when a button is pressed, such as an LED
  • Make the hardware smaller – so that it could fit into other plants or could be portable
  • Make the hardware wireless – so it can work with your iTunes playlist or another array of music files; e.g., it would just give the go ahead that the plant is well watered and you can now play music
  • Use a larger array of sound files, such as from your computer’s music library or a specific playlist you’ve selected. The issue with this improvement is that Minim doesn’t seem to like audio files that are not .wav or .mp3

Flickr Photos!

Play and Next buttons - more photos on Flickr!
Back and Next buttons - more photos on Flickr!

Phys Comp – Week 6: Serial Communication, Part 2

Working with an accelerometer for serial communication
Working with an accelerometer for serial communication

For our second lab exercise with Serial Communication (which I’m doing early because it seems fun), I got to use an accelerometer for the first time. (Borrowed temporarily from another student.)

In the first half of the lab, I was able to get the ball to move around, and added the smooth(); command to Processing to make the edges of the ellipse to smooth out.

In the second half, I had trouble. I actually couldn’t get the call and response connection to work. I’m not sure what happened. I got the hello as we’d programmed from the Arduino, but as soon as I picked up the breadboard and got input from the accelerometer, the serial monitor seemed like it wasn’t sure if it should be outputting bytes or ASCII and put out both numbers and letters that sometimes made sense as words(?).

I’m not sure what the issue was, but hopefully we’ll go over this in class. I’ll raise my hand and ask, for sure.

Phys Comp – Week 6: Serial Communication, Part 1

This week we connected our Arduino to Processing.

When it happened, I was hoped for sparks and for everything to start moving in slow motion. Instead I got a purple graph. It was still pretty excited. I made funny noises.


Breadboard for Serial communication

How to solve a Rubik’s Cube

For the Data Visualization class I’m on the waitlist for, we were asked to come up with the solution for solving a Rubik’s cube.

To solve a Rubik’s cube, you basically need to remember that the whole thing consists of a series of patterns that, if you trust them, will solve the puzzle for you. In the past, when I’ve tried to solve the puzzle, I was somehow able to solve one side of tiles. Then, when I try to complete the rest of the puzzle, I find that I’m stuck. Whenever I try to complete another side, I find that I am too afraid of messing up the side I’ve completed, so I stop working. This fear is, of course, justified, since I do in fact mess up the side I just completed when working on the new side.

I’ve come to learn that what I’ve been missing is an algorithm or several algorithms. “Algorithm” sounds like a math term, and since I tend to stay away from math, I can’t say I’ve come across it before. However, since I’m supposed to come up with a description for how to solve a Rubik’s cube, I did some research:

• I understand that an algorithm is a set of defined instructions that are carried out until certain conditions are met. – Wikipedia

• Different algorithms can be used to accomplish the same task. The one that is used depends on the constraints for each task. –

• Algorithms are used to solve a recurrent problem. –

These are all interesting, and apparently algorithms are what make up computer languages. But, using these definitions, algorithms are also what fill up cookbooks and instruction manuals. Hmmm….

Well, anyway, now for my solution:
1. Get a Rubik’s Cube.
2. Turn the faces of the Rubik’s Cube, one face at a time, until one side consists of the same color in each of the nine squares. Call a side that consists of the same color in each of the nine squares a Full Side.
3. Repeat step 2 for each of the five colors but whenever you need to break a Full Side, so that the nine squares of the same are no longer touching, you must put them back so that they are touching again.
4. Repeat step 3 until there are six Full Sides.