December 12, 2018No Comments

Dialogflow V2: Unity Plugin Prototype

The Dialogflow V2 Unity Plugin is a continuation from Unity Plugin, which made it easy to integrate the Dialogflow( natural language processing API into your Unity project. It's built as a team project in Open Source Studio at ITP and not an official plugin from Google or Unity.

Responsible for project management and documentation. The project is selected for xStory, and will be continued on its development and documentation.

December 5, 2018No Comments

Doge Interactive Type

Doge meme style generator with ML5 PoseNet

Example of

Week 5: Doge style generator using ML5 PoseNet. This is not the best practice to use PoseNet, because a similar effect can be done through random(width), random(height) - not necessarily involving the person's pose. Yet it still makes a difference in a sense that it won't generate any text if it doesn't detect a person inside video. What can possibly done to use pose vertex with text in more apparent way..?

December 5, 2018No Comments

Fonts as Data

Type animation reacting to pitch of sound input

Week 4 assignment based on Drawing ttf with p5js commands + ML5 PitchDetection examples. It's always intriguing to see type itself reacting to sound, but it also makes me question what is the most natural/fitting typographic form to another data, such as: if the current transition of moving left to right goes along with concept of "pitch".

November 14, 2018No Comments

Concrete Poetry(?)

tiled gif image that is related to the spoken words, with red border and red text

tiled gif image that is related to the spoken words, with red border and red text

Week 3 assignment based on Marc Adrian homage and Speech to Gif examples.  Code

November 14, 2018No Comments

The Asemic Letterform

Generating asemic writing in grid - based on Recursive forms (homage to Anders Hoff) example.

October 31, 2018No Comments

The ANSI Terminal

Week 1 assignment based on Ghost train (homage to bpNichol) example. It displays bouncing palindromes that are starting and ending with letter "w", and changes its color and sentence after completing one round. It would be nice to figure out how to make two of them cross each other, just like the original Ghost train example.

October 31, 2018No Comments

Hidden Symbols & Networked Poems

week 1 readings:

The Upright Script: Words in Space and on the Page

  • Ara Shirinyan’s Your Country is Great: Afghanistan–Guyana (New York: Futurepoem, 2008)
    • "knowledge and love of Afghanistan is great even as he regrets the limits of his understanding"
    • This emptiness is clearest in the poems “Burkina Faso is Great,” “Central African Republic is Great,” and “Equatorial Guinea is Great,” each of which contains only a title on a blank page. As Shirinyan notes in his preface, “no one who could write in English and had access to the web thought to say anything great about those countries” in September 2006 when he conducted his searches.
  • Index for X and the Origin of Fires
    • The images continue to refresh, each remaining on screen just long enough to catch the reader’s eye before disappearing, while the text remains onscreen until the next click of the mouse.

Code hidden in Stone Age art may be the root of human writing

World map of

  • Between 2013 and 2014, von Petzinger visited 52 caves in France, Spain, Italy and Portugal.
    • The symbols she found ranged from dots, lines, triangles, squares and zigzags to more complex forms like ladder shapes, hand stencils, something called a tectiform that looks a bit like a post with a roof, and feather shapes called penniforms.
    • In some places, the signs were part of bigger paintings.
  • “Consistent doodles”
    • "The ability of humans to produce a system of signs..."


October 30, 2018No Comments

Colour Contrast Analyser (CCA)

For accessibility assignment, I looked into p5 editor with Colour Contrast Analyser (CCA). The CCA helps to figure out the legibility of text and the contrast of visual elements, along with further useful information like color blindness simulator.

p5 web editor and use of Colour Contrast Analyser (CCA)

The first input I tried on was contrast between grey text and lighter grey(?) background. I used the color picker, and it came out as #B5B5B5 for text (File, Edit, Sketch, Help&Feedback, Auto-refresh...etc) and#FBFBFB for background. Unfortunately, the contrast ratio was 2:1, which fulfill neither "minimum" nor "enhanced" standard for both large and regular text display.

p5 web editor and use of Colour Contrast Analyser (CCA) for point color

The good news is that p5's point color, #EA285F, which is often used for UI components and logo resulted enough ratio to satisfy Contrast (Minimum) for large text, and most importantly, passed the Non-text Contrast. On the other hand, although the same pink color is used for hover text - it still fails for regular text contrast.

October 25, 2018No Comments

Talk to Makeup Guru

Everyday there is another set of new makeup products. Sometime it's about texture, and sometimes it's about brand itself - but most all, it's about color.

"For those of us who are obsessed with makeup — lipstick in particular — there's no such thing as too many of our favorite products. With our laser vision, we can tell the true difference between any two shades, no matter how similar they may look to the untrained eye." — Twitter Boyfriends Are Confused About These Two Obviously Very Different Worlds Apart Lip Colors, KARA NESVIG

Makeup Guru was designed to give "FRESH MAKEUP TUTORIAL: WITH BRAND NEW {color} EYESHADOW!" based on your favorite color at this moment. Original text is from one of the tutorials in Deck of Scarlet. The project is also a continuation of Makeup Guru from Reading and Writing Electronic Text class, but in more conversational form.


example of using Makeup Guru when you choose pink lavender color



To generate entities and functions, I used xkcd color data from Corpora. Full code can be found here: Makeup Guru. The process is composed of (1) sorting hex to RGB (2) building word vectors based on RGB values (3) receiving user's favorite color and seeking nearest 6 colors (4) implementing colors to tutorial along with other elements.

example of using Makeup Guru when you choose one of green shades

example of using Makeup Guru when you choose pink gross green color

xkcd.json provides a wide variety of 952 colors, and is inclusive to such colors as poo, baby puke green, snot shade, diarrhea... etc. Of course, beauty is subjective (especially when it comes to an abstract component as color), so some people might want those shades in their makeup. Regardless of the terms, the Makeup Guru will go through the same process and give you the tutorial.


1. Hex to RGB

def hex_to_int(s):
    s = s.lstrip("#")
    return int(s[:2], 16), int(s[2:4], 16), int(s[4:6], 16)
colors = dict() returns the same xkcd colors along with RGB values as sorted.json, which looks like this:

  "cloudy blue": [172, 194, 217],
  "dark pastel green": [86, 174, 87],
  "dust": [178, 153, 110],
  "electric lime": [168, 255, 4],
  "fresh green": [105, 216, 79],
  "light eggplant": [137, 69, 133],


2. Color Vectors

The original codes I have are in python and p5.js (as separate sketches), which were relatively easier to create vector space. In order to adapt the same structure, I used vectors node module. var dist = require('vectors/dist')(3) (and it only operates with var) creates three dimensional vector space, which can be used for locating [red, green, blue] values.

function findNearest(v) {
  var dist = require('vectors/dist')(3)
  let keys = Object.keys(colorVector);
  keys.sort((a, b) => {
    let d1 = dist(v, colorVector[a]);
    let d2 = dist(v, colorVector[b]);
    return d1 - d2;
  nearestColors.length = 0;
  for (let i = 0; i < 7; i++){

fineNearest() seeks 7 nearest colors from the "favorite color" and results nearestColors - these 7 colors are used for generating the overall tutorial text.


3. Favorite Color and Nearest Colors

app.intent('Pick Color', (conv, params) => { = params.xkcd;
  pos = colorVector[]

For example, if you picked "cocoa" as your favorite color, than it will be:

  pos = colorVector["cocoa"]
  findNearest([135, 95, 66])


4. Colors in Tutorial

    `How exciting! To give you a bold look, I'm switching to a bigger blending brush.` +
    ` And I’ll use it to blend out the edges of that ` +
    nearestColors[3] + ` shade into the crease and, you know, make it look really nice and seamless.` +
    ` And see how that eyeshadow kind of blended into more of like a ` +
    nearestColors[4] + ` shade? When it’s fading, it looks more ` +
    nearestColors[5] + `, rather than ` + nearestColors[6] + `.` +
    ` It is really nice. It looks like I used many different colors when I just used one single eyeshadow.`


Further Notes 💄

  • Sometime I think marketing operates in similar way as fortune teller: consumers give the "seed" of their story, and fortune tellers/industries rephrase and repackage the content that is given by the consumers. So you are actually offering them a hint to target you.
  • Throughout the tutorial, the Makeup guru talks about only one shadow product, in many different phrases.
  • node-word2vec might be interesting to use for future related projects.

October 17, 2018No Comments

Anatomy of an AI System

Anatomy of an AI System map

Anatomy of an AI System By Kate Crawford and Vladan Joler

  • designed to either “blend in or stand out”
  • “Because Alexa is in the cloud, she is always getting smarter and adding new features.”
    • hard to explain and understand the extraordinary complexity of these artificial intelligence agent
    • but do people care?
    • requires “a vast planetary network, fueled by the extraction of non-renewable materials, labor, and data”
    • reminds me Sandra
  • lithium extraction
    • Atacama regions in Chile and Argentina
    • ‘grey gold’
    • lithium-Ion batteries
    • limited lifespan
    • invisible threads of commerce, science, politics and power
  • three processes: material resources, human labor, and data
    • the ethereal metaphor of ‘the cloud’
    • it’s hard to ‘see’ any of these processes individually
  • human user = chimera (hybrid)
    • a consumer, a resource, a worker, and a product
    • aren’t we always a ‘chimera’ in real life as well?
    • helping to train the neural networks
    • ‘collective intelligence’?
  • The echo = an ‘ear’ in the home
  • statua citofonica (the ‘talking statue’)
    • “listening systems” = power, class, and secrecy