Advanced Interactive Design - Project 2
Kyra Binti Rizal Hamzah / 0337085 /Advanced Interactive Design / March-July 2021
PROJECT 2
Requirements:
- use face tracker
- use UI picker (minimum 3 options)
- background replacement
- use face interaction patch
Project 2 topic / theme is open, you can do anything you want, Just make something you enjoy and feel fun to do.
Week 6
The idea I had in mind for this project was to do a sort of day/night theme.
As the requirement was to have at least 3 options, I wanted to do 1 option that would be the "interactive" one - where you could do a movement and the filter would switch from being day mode to night mode. For the other two options, I planned to make them only the face details of each "mode" (one for day, one for night) without the background , so you could have different levels of becoming one with the "artwork"
 |
| I didn't follow this super closely, but this was a reference that I had in mind - I wanted to bring the clouds/rays/smoke in the background and foreground into my filter. Also, the colorful details on the face inspired me in regards to actually making this a face filter. |
As the requirement was to have at least 3 options, I wanted to do 1 option that would be the "interactive" one - where you could do a movement and the filter would switch from being day mode to night mode. For the other two options, I planned to make them only the face details of each "mode" (one for day, one for night) without the background , so you could have different levels of becoming one with the "artwork"
After that it was sketch time!!
 |
| I like starting off with really rough pencil sketches to focus on the main form and placement and to play with ideas for different aspects to include before I proceed with the actual drawings (I eventually ditched the idea of having a crown/floating elements around the head |
 |
| Initial illustration for the night version |
 |
| An even earlier progress shot that featured a more asymmetrical design. |
As my initial idea involved a sort of drawing that made a border around the whole eye, it was very difficult to actually use and have it look good on different faces. Given that different people have different eye shapes, it would warp in a lot of different ways when I tested it on myself or on the preloaded videos of different people in SparkAR. I then made the decision to make the design less "tight" around the eye, and redrew it, keeping in mind that I should leave some space around the eyelid, so it could fit on different faces.
One thing I noticed as well, was the nature of the face mapping in general - I have a bit of a flat face, which is pretty commonplace on Asian faces in general. Thing is, the face map follows a much more eurocentric face shape as evident in the various templates given. So when I tried the filter I found it would wrap in parts of my face as if there was a curve there, when reality is my face was still flat there. Honestly I could see this same thing to different extents on some of the faces built in SparkAR as well, as the video models in the software did contain models of different ethnic backgrounds. Something to keep in mind, and just some food for thought as well. Carrying on now.
 |
| The revised version as well as the face asset for day mode - I kept the right side similar (same cloud shape recoloured) and put more variation on the left side. |
Honestly the bulk of my time was spent on this - readjusting the shape of the eye details so that it would work well on different faces. It was quite a challenge to draw something meant for 3d space within a 2d space - this is my first time doing something of the sort.
The rest of the illustrations were a piece of cake as I could keep them 2d!!!
 |
| Illustrations for background and foreground (the matching cloud on the bottom of the screens in pink on day and yellow on night are foreground details) |
Got a lot done the previous week so I took this week as a break since raya was this week anyway
Week 8
Needed to start assembling the filter in SparkAR this week, adding the different UI options and interactions, so I started by creating the icons to be used for the UI Picker - This didn't take long as I just rearranged different parts of the illustrations into a circle frame
 |
| Left to right - Interactive nod to change option with both illustration themes, night mode, day mode |
The remaining work was to load everything into SparkAR and add the patch nodes.
I started with making UI options 2 and 3 (The non interactive ones) as they were the easiest to put together ; all I had to do was connect the elements to the option sender node.
 |
| The calm before the storm. |
The first option (interactive) was a lot harder to put together. Typically, one would use the face tracker as a way to navigate between different options. Not to switch assets within one singular option. I tried many things, pored through the SparkAR documentation on their website, and just did a lot of trial and error that I was too absorbed in trying to succeed in to remember to stop and take progress screenshots, before I came to an answer.
The issue was that while I could make the face tracker switch the assets, I couldn't do that while having a separate option picker running as well. Basically, the two would end up overlaying, and there isn't a way to disable the interactions within specific UI options.
Am I making sense?
Anyways, the workaround is a bit.. funny. It's either super brilliant or super stupid and unintuitive. You be the judge.
Anyways, the workaround is a bit.. funny. It's either super brilliant or super stupid and unintuitive. You be the judge.
See, when arranging elements in SparkAR, you need to make sure you put the Canvas element (used for background images and such) beneath the face tracker layer and not above it - if you place it above, the face tracker does not work. Aha. Exactly what I wanted to accomplish. So, I added a Canvas above the face tracker set up for the interactive option. I made it invisible upon launch, but then visible upon changing to the non-interactive options.
Long story short, Invisible=Face tracker works. Visible=Face tracker doesnt work. Rather than disabling the face tracker I simply made it hidden when not necessary.
(A non interactive face tracker was still necessary for the face details on the other options so for those I added a separate face tracker that was placed above the canvas)
 |
| Finished patch nodes - Picker UI |
 |
| Finished patch nodes - face tracker; nod to change from day to night (and vice versa) |
Demo of the finished filter!
