December 12, 2018No Comments

Dialogflow V2: Unity Plugin Prototype

The Dialogflow V2 Unity Plugin is a continuation from Unity Plugin, which made it easy to integrate the Dialogflow( natural language processing API into your Unity project. It's built as a team project in Open Source Studio at ITP and not an official plugin from Google or Unity.

Responsible for project management and documentation. The project is selected for xStory, and will be continued on its development and documentation.

December 5, 2018No Comments

Doge Interactive Type

Doge meme style generator with ML5 PoseNet

Example of

Week 5: Doge style generator using ML5 PoseNet. This is not the best practice to use PoseNet, because a similar effect can be done through random(width), random(height) - not necessarily involving the person's pose. Yet it still makes a difference in a sense that it won't generate any text if it doesn't detect a person inside video. What can possibly done to use pose vertex with text in more apparent way..?

December 5, 2018No Comments

Fonts as Data

Type animation reacting to pitch of sound input

Week 4 assignment based on Drawing ttf with p5js commands + ML5 PitchDetection examples. It's always intriguing to see type itself reacting to sound, but it also makes me question what is the most natural/fitting typographic form to another data, such as: if the current transition of moving left to right goes along with concept of "pitch".

October 31, 2018No Comments

Hidden Symbols & Networked Poems

The Upright Script: Words in Space and on the Page

  • Ara Shirinyan’s Your Country is Great: Afghanistan–Guyana (New York: Futurepoem, 2008)
    • "knowledge and love of Afghanistan is great even as he regrets the limits of his understanding"
    • This emptiness is clearest in the poems “Burkina Faso is Great,” “Central African Republic is Great,” and “Equatorial Guinea is Great,” each of which contains only a title on a blank page. As Shirinyan notes in his preface, “no one who could write in English and had access to the web thought to say anything great about those countries” in September 2006 when he conducted his searches.
  • Index for X and the Origin of Fires
    • The images continue to refresh, each remaining on screen just long enough to catch the reader’s eye before disappearing, while the text remains onscreen until the next click of the mouse.

Code hidden in Stone Age art may be the root of human writing

World map of

  • Between 2013 and 2014, von Petzinger visited 52 caves in France, Spain, Italy and Portugal.
    • The symbols she found ranged from dots, lines, triangles, squares and zigzags to more complex forms like ladder shapes, hand stencils, something called a tectiform that looks a bit like a post with a roof, and feather shapes called penniforms.
    • In some places, the signs were part of bigger paintings.
  • “Consistent doodles”
    • "The ability of humans to produce a system of signs..."


October 30, 2018No Comments

Colour Contrast Analyser (CCA)

For accessibility assignment, I looked into p5 editor with Colour Contrast Analyser (CCA). The CCA helps to figure out the legibility of text and the contrast of visual elements, along with further useful information like color blindness simulator.

p5 web editor and use of Colour Contrast Analyser (CCA)

The first input I tried on was contrast between grey text and lighter grey(?) background. I used the color picker, and it came out as #B5B5B5 for text (File, Edit, Sketch, Help&Feedback, Auto-refresh...etc) and#FBFBFB for background. Unfortunately, the contrast ratio was 2:1, which fulfill neither "minimum" nor "enhanced" standard for both large and regular text display.

p5 web editor and use of Colour Contrast Analyser (CCA) for point color

The good news is that p5's point color, #EA285F, which is often used for UI components and logo resulted enough ratio to satisfy Contrast (Minimum) for large text, and most importantly, passed the Non-text Contrast. On the other hand, although the same pink color is used for hover text - it still fails for regular text contrast.

October 25, 2018No Comments

Talk to Makeup Guru

Everyday there is another set of new makeup products. Sometime it's about texture, and sometimes it's about brand itself - but most all, it's about color.

"For those of us who are obsessed with makeup — lipstick in particular — there's no such thing as too many of our favorite products. With our laser vision, we can tell the true difference between any two shades, no matter how similar they may look to the untrained eye." — Twitter Boyfriends Are Confused About These Two Obviously Very Different Worlds Apart Lip Colors, KARA NESVIG

Makeup Guru was designed to give "FRESH MAKEUP TUTORIAL: WITH BRAND NEW {color} EYESHADOW!" based on your favorite color at this moment. Original text is from one of the tutorials in Deck of Scarlet. The project is also a continuation of Makeup Guru from Reading and Writing Electronic Text class, but in more conversational form.


example of using Makeup Guru when you choose pink lavender color



To generate entities and functions, I used xkcd color data from Corpora. Full code can be found here: Makeup Guru. The process is composed of (1) sorting hex to RGB (2) building word vectors based on RGB values (3) receiving user's favorite color and seeking nearest 6 colors (4) implementing colors to tutorial along with other elements.

example of using Makeup Guru when you choose one of green shades

example of using Makeup Guru when you choose pink gross green color

xkcd.json provides a wide variety of 952 colors, and is inclusive to such colors as poo, baby puke green, snot shade, diarrhea... etc. Of course, beauty is subjective (especially when it comes to an abstract component as color), so some people might want those shades in their makeup. Regardless of the terms, the Makeup Guru will go through the same process and give you the tutorial.


1. Hex to RGB

def hex_to_int(s):
    s = s.lstrip("#")
    return int(s[:2], 16), int(s[2:4], 16), int(s[4:6], 16)
colors = dict() returns the same xkcd colors along with RGB values as sorted.json, which looks like this:

  "cloudy blue": [172, 194, 217],
  "dark pastel green": [86, 174, 87],
  "dust": [178, 153, 110],
  "electric lime": [168, 255, 4],
  "fresh green": [105, 216, 79],
  "light eggplant": [137, 69, 133],


2. Color Vectors

The original codes I have are in python and p5.js (as separate sketches), which were relatively easier to create vector space. In order to adapt the same structure, I used vectors node module. var dist = require('vectors/dist')(3) (and it only operates with var) creates three dimensional vector space, which can be used for locating [red, green, blue] values.

function findNearest(v) {
  var dist = require('vectors/dist')(3)
  let keys = Object.keys(colorVector);
  keys.sort((a, b) => {
    let d1 = dist(v, colorVector[a]);
    let d2 = dist(v, colorVector[b]);
    return d1 - d2;
  nearestColors.length = 0;
  for (let i = 0; i < 7; i++){

fineNearest() seeks 7 nearest colors from the "favorite color" and results nearestColors - these 7 colors are used for generating the overall tutorial text.


3. Favorite Color and Nearest Colors

app.intent('Pick Color', (conv, params) => { = params.xkcd;
  pos = colorVector[]

For example, if you picked "cocoa" as your favorite color, than it will be:

  pos = colorVector["cocoa"]
  findNearest([135, 95, 66])


4. Colors in Tutorial

    `How exciting! To give you a bold look, I'm switching to a bigger blending brush.` +
    ` And I’ll use it to blend out the edges of that ` +
    nearestColors[3] + ` shade into the crease and, you know, make it look really nice and seamless.` +
    ` And see how that eyeshadow kind of blended into more of like a ` +
    nearestColors[4] + ` shade? When it’s fading, it looks more ` +
    nearestColors[5] + `, rather than ` + nearestColors[6] + `.` +
    ` It is really nice. It looks like I used many different colors when I just used one single eyeshadow.`


Further Notes 💄

  • Sometime I think marketing operates in similar way as fortune teller: consumers give the "seed" of their story, and fortune tellers/industries rephrase and repackage the content that is given by the consumers. So you are actually offering them a hint to target you.
  • Throughout the tutorial, the Makeup guru talks about only one shadow product, in many different phrases.
  • node-word2vec might be interesting to use for future related projects.

October 17, 2018No Comments

Anatomy of an AI System

Anatomy of an AI System map

Anatomy of an AI System By Kate Crawford and Vladan Joler

  • designed to either “blend in or stand out”
  • “Because Alexa is in the cloud, she is always getting smarter and adding new features.”
    • hard to explain and understand the extraordinary complexity of these artificial intelligence agent
    • but do people care?
    • requires “a vast planetary network, fueled by the extraction of non-renewable materials, labor, and data”
    • reminds me Sandra
  • lithium extraction
    • Atacama regions in Chile and Argentina
    • ‘grey gold’
    • lithium-Ion batteries
    • limited lifespan
    • invisible threads of commerce, science, politics and power
  • three processes: material resources, human labor, and data
    • the ethereal metaphor of ‘the cloud’
    • it’s hard to ‘see’ any of these processes individually
  • human user = chimera (hybrid)
    • a consumer, a resource, a worker, and a product
    • aren’t we always a ‘chimera’ in real life as well?
    • helping to train the neural networks
    • ‘collective intelligence’?
  • The echo = an ‘ear’ in the home
  • statua citofonica (the ‘talking statue’)
    • “listening systems” = power, class, and secrecy

October 9, 2018No Comments

Dialogflow: Fulfillment and Logic

Sketchbook gif from Don't hug me I'm scared

Assignment 4 is a practice based on Codelabs (part 1 and part 2) for building Actions for the Google Assistant. It contains elements as Permission, Suggestion, and BasicCard along with usage of webhook.

It is a parody of Don't Hug me I'm Scared.

October 1, 2018No Comments

Sandra Podcast

Sandra Podcast Image

Sandra podcast

  1. episode 1 (Hope is a Mistake, 17:59)
  2. episode 2 (The User Experience, 21:51)

Sandra, the series of podcast episodes about this virtual assistant that is actually powered by an army of real people, reminded me back to one of the second week’s reading: The Difference followed by the author’s note. In the comments, many readers were debating either if this is a human being or a chatbot who is answering in the chatroom. My favorite comment was by Loki: “…But if he is human, than this is a great story to show how possible it is to program someone. Keep telling someone that they aren’t a person and they’ll eventually start to believe it themselves.”

Therefore, Sandra is not all about faking those sleek and mighty technology products but also about identity. Who’s containing who? In this podcast series, it seems like Helen is “inside” the Sandra machine - making Sandra as a mere container. Why can’t Sandra just sound like Helen, like a normal human? It also resonates well with the other podcast episode, Helpful Mom Voices, which talks about nowadays TTS doesn’t need to “appear” as human but it still attempts to bring high technology “down to human level.” In case of Sandra, it’s the opposite. It seems like as soon as the answer/reaction from the humans behind Sandra is transformed into the typical AI assistant voice, people seem to feel comfortable - even too much comfortable to ask improper and impolite queries.  It’s such an intriguing storyline as a podcast, but I hope that no one actually comes up with the same idea.

October 1, 2018No Comments

Dialogflow: Intents, Entities and Contexts

Assignment 3 that the user plays adventure game only through simple Q&A. It's built with Dialogflow but needs some revision due to my lack of understanding in concepts of entities and contexts.

It was inspired by text-based role-playing game such as Candy Box! and A Dark Room.