Time-of-flight cameras and displacement maps

These days I've stumbled upon a very interesting topic: Time-of-flight cameras.

I heard that Microsoft is using them to capture 3d coordinates of players on the amazing Project Natal.

I'm on a project (hobby) that needs that. I don't want to have two cameras to capture users from front and side but instead a single front camera which would know the depth of stuff I need.

Time-of-flight cameras are useful in this matter since they locate points on 3d space according to the light analysis. For more information plaese refer to http://en.wikipedia.org/wiki/Time-of-flight_camera, or any other topic about that on the Internet.

When I was playing around with 3ds Max some time ago I verified a very interesting feature of it called: Displacement Maps.

Quickly, displacement maps provide a mechanism to create a surface where dark areas of the picture (map) will create valleys and brighter areas will represent the mountains. This is: the darker, the deeper and the brighter the higher.

I made some tests with a picture of me with some gloves I got and the result was exciting (check below). I expect to work on that and implement it on C# with AForge help and in future XNA.

See you next time,

1. Original picture

2. Original picture loaded on a plane.

3. Plane with Displacement applied (perspective view).

Computer Vision

Hey!

It's been a while since my last post, I was in a bunch of compromises and I decided to travel last two weekends to loose, my routine was mind sucking and talk to some great people I haven't met for some months and I must say that this quick chunking recharged my batteries.

I don't know why but I decided to take a break on the 3D Modeling but that I mentioned on the last posts, I think I crossed against a small project that I thought I could conclude on a single week but this project is taking more than that.

I'm using C# and AForge (computer vision library for C#) in order to do body tracking and collision with virtual objects in real-time video. I got stuck on some Asynchronous problems and Marshalling but I could move on the problem I'm getting now is a very understandable one: performance.

Imagine a 30 fps video file coming to you and you want to apply all filters on it. You have 33 ms per function to do everything before a new frame comes in (1 second = 1000 ms; time per function = 1000 ms / 30 = approx. 33 ms). I'm getting 222 ms per function, it won't be an easy job to do that in C# (I know C/C++ would be faster, but I would've to waste many time on UI and things like that and AForge provides lots of great functions and support that made me use C#).

I'm posting the current result of my processing, these images are completely test and won't be used on the final project, so just pay attention on the idea itself which is: remove the background and provide corner detection for future usage on collision detection.

This is the original image.

This is the image after background removal and corner detection. Unfortunately the corner detection still have to be worked - it is not detecting correctly due the brightness it uses to detect corners, so I have to decrease that a bit so the corners can be seen by the algorithm. Talking about that, if someone would like some directions on corner detection algorithm I'd recommend Susan Corners, which I'm using here (provided by AForge).

And that's it.
Next days I intend to finish this and resume the 3D Modeling book plus study more to the Master appliance test on Electrical Engineering.

See you,
Caio.

Cloud Computing

These days, where broadband gets faster enough to stream information in a faster way, some interesting applications arise. These days, two of them called up my attention, they are:

eyeOS:
http://eyeos.org/

Sumo Paint:
http://www.sumo.fi/products/sumopaint/index.php?id=0

Cloud Computing is really a fact which cannot be simply ignored. Don't you know what Cloud Computing is? Google is a great example of that. You don't know where your mail data is stored, sometimes they are stored in different places around the world but it shows to you (the final user) as a single entity (transparency property). Nice, eh?

3D Modeling

Last week arrived my book on 3D Modeling. A very nice book, which I would recommend for everyone which wants to learn fast 3D stuff. The book is called 3D Modeling for Games from Andrew Gahan. It comes with a great DVD which is used on the whole book.

After I bought this book I joined the board below:
http://www.3d-for-games.com/forum/index.php

There are lots of nice and skilled people there, Andrew (Andy) is also there and can help anyone - for free, kidding :P

I'm still reading the book so I intend to post things related to it while I do the exercises and things like that.

IT Governance

Last month I presented my MBA thesis on IT Governance. It was very gratifying to finish the course, it was very helpful for my professional career although I'm not putting this in practice on my current job I certainly will do in future.

This topic was created because today I finished reading Peter's Weill & Diana's Ross IT Governance book. The book covers some interesting topics on IT Governance focusing on many different aspects without bind the reader to any of the existent frameworks or technologies but pointing what should be done in order to make IT support the company's strategy.

Now I'm starting to read an algorithms book. But before someone asks me, I say: Yes I know how to program. The thing is that I'm studying for a Master exam which will be happening by the end of this year, so I expect to pass on it and since this book was recommended, why not read it? It seems very complete on this area. Book name: Data structures and their algorithms - authors: Jayme Luiz Szwarefiter & Lilian Markenzon.

The Master program is about Human Computer Interaction mostly, nice isn't it? Hope I can study everything needed until October :P

Introduction

When I was young I though that I loved games* in general but after I grew up I verified that not only the games are nice but virtual reality is also a nice topic in Computer Science and it doesn’t have to be a game to be fun: augmented reality is an example (I expect to talk about that on another topic).

I grew in the 80’s, so yes, I played the ATARI, passing to the SEGA’s Mega Drive, Nintendo’s unforgivable Super Nintendo, and the Sony Playstation with its astonishing 3D graphics.

I’d be unfair if I didn’t mention the PC games, they’ve always accompanied me parallel to consoles.

I thought of computer as something mysterious, a simple box a complete interaction but not a finished experience, something that every year evolves and changes the way people work, behave and conduct their lives. How a box can bring you fun, information, relationships and many things in such diversity. How people put such things inside of it? How it works?

With all those things in mind I decided study Computer Science.

I’m not going to describe the whole course, but it was very important for me, I discovered that as a woodworker can create useful benches I could create useful softwares and that these softwares would depend on what I want it to do.

To create softwares you need knowledge about many things, for example, if you want to create a software about water treatment you should know not only what water treatment is but how that works and how companies of this segment are work. Besides that, one thing is indispensable: a programming language. A programming language instructs the computer to perform some actions according to your needs.

Nevertheless, along the course I learned many interesting technologies, one of them was mobile development. I watched a lecture about a guy which was talking about something very recent at that moment, it was the .NET framework. He was saying that it was a powerful technologies and one could create many applications easily, also for mobile platforms, such as cell phones, PDAs, Palms, etc. In that moment, my curiosity awoke. I remembered the old times where I had some mini-games devices (bought by my father after I cried a lot to him – thanks dad) so I though that creating games for cell phones would be the next step in games.

I could not imagine but that was the start of my journey, the first spark. I decided learning how to program in mobile devices, that’s where I came across BREW a C/C++ API for CDMA cell phones, mine was one, so I decided to learn it. I had to learn many things, I read two books about it and created a project to put the experience on practice, this was my first game. Successfully I could commercialize this game on Venezuela after mailing many publishers.

I got a job on that area, which was very gratifying. I felt the value of my sweat and all effort that I had put to make this happen.

Unfortunately, here in Brazil companies are very small and that time I was looking for a better job so I changed again, this time not for a game company, but the passion for games is still there.

Since then, I’ve been studying computer games by my own, and I’m crossing against many interesting topics which I want to share with people and that is the main reason I’m creating this blog. So if you feel somehow similar to me, do not hesitate in following it, we can share some good experience over this area.

These days, I verified that being a computer scientist is different than a programmer. A computer scientist has a compromise with science as the name says. In my understanding a computer scientist should research at least whilst a programmer needs to make the computer perform what is needed by the project.

I intend to focus on Augmented, Mixed Reality; 3D Modeling; Programming; Human Computer Interaction and new technologies.

Thanks for reading,

See you soon

*When I say games I refer to computer games and console games