This is my first project for the CS491 Virtual and Augmented Reality class at UIC. It consisted in creating an Augmented Reality breakfast table, with some given targets: a placemat, cereal boxes, cans, magazine. The goal was to create a scene of how breakfast could be in the future, by augmenting everything with 3D objects, particles, sounds and implementing interactions between objects. I chose the theme of Alice in Wonderland because it's the first thing that came to my mind when the professor was explaining the project specifications.
Stay with me in Wonderland, and I'll show you how deep the rabbit hole goes.
Download entire project
The custom target can be downloaded here
Please notice that I modified the default method onTrackableStateChanged of the TrackableEventHandler class to signal to my scripts when specific trackable objects are lost
MODELS
This is a list of the models I took from the web and imported into my application without modifying them at all, you can click on the model image to go to the source website and download the model.
MY MODELS
This is a list of the models I created from scratch or got from the web and modified, such modifications include: mesh transformation into low-poly mesh for the purpose of reducing the size and the graphic computation to render the scene, exploding object and taking pieces of it, UV Unwrapping and creation of textures to color models that had no material. If a model comes from somewhere in the web the link will be its source page, and the modified object can be found by downloading my entire project. Models created from scratch by me have a link to download the model in formats of type .FBX, .obj or .dae, complete with textures.
These are the sounds source associated with each of the objects.
Main scenes
Scene I
Mad Hats cereal box
The first cereal box contains yellow cereals in a hat shape, in the Augmented Reality scene you can see them floating around the cereal Box and going from the 2D bowl on the back in the 3D bowl in the front.
The 3D objects in the scene are: the mad hatter, another one of his beautiful hats, a pair of scissors he needs to build hats, a bowl, a spoon and the cereals.
I entirely created the bowl and the cereals, while for the mad hatter I took a High Poly model, transformed it in a low Poly, UV-unwrapped the 12 pieces of which it's made and created a texture for each one. The faces of the box are completely replaced by 2D graphics.
This cereal box interacts with all the other boxes and the can, when it's close to one of them the mad hatter starts saying advertising for his cereal creations. It also interacts with the placemat, that displays nutrition facts as soon as it is placed onto the placemat, and as explained in the Placemat interactions section, it could trigger a sad or happy 3D emoji.
The cereals sound plays when the box is put onto the placemat.
Scene II
Smokey Os cereal box
The second cereal box contains grey donut-shaped cereals, in the Augmented Reality scene you can see them been blown out from the mouth of the 2D Caterpillar which is smoking Shisha on the front. The smoking cereals in the animation tend to disappear over time thanks to the diminishing alpha component of the faded material applied to the object.
The 3D objects in the scene are: animated green and blue mushrooms, red and white mushrooms, white smoke (particle object), the animated cheschire cat head, a duck-vinegar object, a caterpillar on the back, the cereals created with a displacement modifier in order to present a rough surface.
I entirely created the cereals and the white and red mushrooms, the faces of the box are completely replaced by 2D graphics.
The smokey Os box interacts with the mad hats and when they're close enough a "we're all mad here" 3D text by the cheshire cat starts to appear and fades away. It also interacts with the placemat, that displays nutrition facts as soon as it is placed onto the placemat, and as explained in the Placemat interactions section, it could trigger a sad or happy 3D emoji.
The smoke sound plays when the box is put onto the placemat.
Scene III
Aced Tea
The augmented can became a iced tea beverage called Aced Tea since its main theme is the Aces cards from Alice in wonderland.
The 3D objects in the scene are: the door handle, the animated card that is spat out from the mouth of the door handle, the ace cards, one of which's spear is animated, a mug and a floating "drink me" sign. I entirely created the mug and its texture, the faces of the box are completely replaced by 2D graphics.
The aced tea interacts with the mad hats and when they're close enough the "Drink me" signs starts to have a periodic movement. It also interacts with the placemat, that displays nutrition facts as soon as it is placed onto the placemat, and as explained in the Placemat interactions section, it could trigger a sad or happy 3D emoji.
The tea kettle sound plays when the can is put onto the placemat.
Scene IV
Rabbit Hole
The rabbit hole is a special food container, it contains a random sample of a type of food.
The 3D objects in the scene are: the rabbit hole, a pencil-animal, a coke bottle, a jam jar, the rabbit with the animated "Follow me" text. I created the abstract rabbit hole in the form of a black hole.
This cube interacts with the mad hats and when they're close enough the rabbit hole behaves like a black hole, swallowing the objects around it.
The engine sound of the rabbit hole machine plays when it's swallowing objects.
Scene V
The Alice in wonderland Cookbook
The Alice in wonderland Cookbook is ideally a recipes book inspired by Alice in wonderland, you can use the virtual buttons, displayed in the pages as logos of an open book and closed book, to flip pages back and forth.
The Book also contains instructions on what your healthy breakfast should be like. In particular, it says that it's okay to eat any of the two types of cereals with aced tea, but it's not recommended to eat both cereals as your meal would be hypercaloric. This is also reminded by the placemat, as soon as you place your chosen items on it, a smiley or sad face will appear depending on your choice. Put any of the cereal boxes and the can on the placemat to make the 3D smiley emoji appear, put both the cereals onto the placemat to make the sad emoji appear instead.
Interactions among objects
The mad hatter drives everyone crazy, just put any of the other boxes or can around him and magic animations will start.
Interaction I
Mad Hats and Aced Tea
When the mad hatter is close enough to the Aced Tea, he will start screaming and advertising for his cereal creations. On the other hand a "Drink me" sign starts floating with a periodic movement.
Interaction II
Mad Hats and Smokey Os
When the mad hatter is close enough to the Smokey Os, he will start screaming and advertising for his cereal creations. On the other hand the cheshire cat will start saying "we're all mad here" in a 3D text which eventually fades away.
Interaction III
Mad Hats and Rabbit Hole
When the mad hatter is close enough to the Aced Tea, he will start screaming and advertising for his cereal creations. On the other hand, the rabbit hole will start to behave like a black hole and will eat all objects around it. The sound of an engine will also start playing in this occasion.
Placemat interactions
Interaction I
Cereals and can with placemat
As soon as a cereal box or the can are placed on the placemat, a display with nutritional information relative to the specific object appears, the quick sound of the holographic display is a laser sound. Moreover, the cereal boxes and the can will start playing their sounds: the mad hats' sound is the one of crunchy and yummy cereals, the Smokey Os' is that of the caterpillar smoking, while the Aced tea's one is that of a tea kettle whistling.
Interaction II
Smile emoji by eating any cereal box and the can
As explained in the Alice in Wonderland CookBook/"Magazine", the placemat reacts based on the type of food the user is trying to eat, in particular, a good breakfast suggestion is to have any of the cereal boxes along with a good mug of Aced Tea. In this case a smiling 3D emoji will set as a sun on the background.
Interaction III
Sad emoji by eating both types of cereal
As explained in the Alice in Wonderland CookBook/"Magazine", the placemat reacts based on the type of food the user is trying to eat, in particular, having both the cereal boxes together is a bad meal because it would consist in too many calories. In this case a sad 3D emoji will set as a sun on the background.
During the development
Some screenshots during the development and the final picture.
Screenshot I
UV Unwrapping on Mad Hatter
Screenshot II
Painting the texture of the smile emoji
Screenshot III
Painting the texture of the Smokey O cereal
Screenshot IV
Final picture
Augmented breakfast in the future
This is a discussion on how an augmented breakfast could be of use in the future23rd September 2018
It is likely that Augmented Reality will become pervasive in the near future and start appearing in every context. It will certainly be with us on the dining table, and we’ll wake up with an augmented reality breakfast every morning. This could be a very interesting situation because they say that breakfast is the most important meal of the day and it’s very crucial to eat properly in order to start off our day in an optimal way. What if we could have a little help? An augmentation that would show us what is better to eat every day, what to mix together to reach the perfect amount of carbohydrates and vitamins to be ready to face the day with enough energy.
One of the big problems that leads to a hasty and unhealthy breakfast is time, no one in the morning has time to think what to eat, to look at nutrition facts and try to calculate the calories based on the amount of product consumed, the only solution would be to use a memoization technique, pre-compute and store the exact amount of each product to consume each morning. But what if for some reason you wake up and you want to change your ordinary breakfast or integrate it with something else? This would complicate things.
What could be helpful for this problem is an augmented reality system, embedded in our glasses or contact lenses which is able to locate and recognize any type of food and display to us in real time how much of that to eat, suggest us what to eat it with and how to mix it to prepare an impeccable quality dish. As soon as any product is put on our placemat information such as nutrition facts appear in front of us and we cannot ignore it as much as we would like to, this could really help us start eating healthier.
Another interesting feature could be the possibility to keep track of what the user is eating throughout the day, showing total calories and set small milestones to reach during each day, suggesting each time what to eat next to be able to reach the needed nutrients. The application could also be integrated to other types of applications and for instance monitor the user’s vital signs and modify its behavior based on that, or it could create a shopping list suggestion for the user as soon as he opens the fridge and anything the user is used eat is noticed to be missing by the system.
All this could be achieved by being able to recognize and track every type of food. This seems like an impossible task nowadays, but in the future this could be possible, the integration of a quick look-up functionality on the recognized food using deep learning could help retrieve in real time the information needed, and the act of eating itself is recognizable by where the object is going and where it disappears.
A thing that could potentially ruin everything about this application are the companies that sell food, if they’ll be given the possibility to buy augmenting ads and appear in front of the user as soon as something like a competitor’s product triggers them, and start screaming something to try and make the user switch for their product instead. This could become a nightmare for the users and they would try in every possible way to get rid of this application, losing all the benefits it could bring them. But let’s be positive and hope that this doesn’t happen, because honestly, I cannot wait to see this type of technology with us every day.