March 5, 2019No Comments

Image Feature Extraction

Based on image-search.ipynb and image-tsne.ipynb

  • Solve the alpha channel issue for feature extraction; directly pull images from the API
  • "This is due to the fact that eps does not know about transparency and the default background color for the rasterization was (0, 0, 0, 0) (black which is fully transparent)." --- continuing error in analysis + expanding features
def load_image(path):
img =
img = scipy.misc.imresize(np.array(img), (224, 224), interp='bicubic')
img[:,:,0] = 255.0-img[:,:,3]
img[:,:,1] = 255.0-img[:,:,3]
img[:,:,2] = 255.0-img[:,:,3]
img = img[:,:,:3]
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
return img, x
<ipython-input-22-6fd16072f6c2> in load_image(path)
8 img =
9 img = scipy.misc.imresize(np.array(img), (224, 224), interp='bicubic')
---> 10 img[:,:,0] = 255.0-img[:,:,3]
11 img[:,:,1] = 255.0-img[:,:,3]
12 img[:,:,2] = 255.0-img[:,:,3]

IndexError: index 3 is out of bounds for axis 2 with size 3
  • Test with more than thousands images (max 50 per a call)
auth = OAuth1("API KEY", "API KEY2")
endpoint = "{term}?page=2"
response = requests.get(endpoint, auth=auth)
with open('./bicycle-p2.json', 'w') as results_file:
json.dump(response.json(), results_file)

February 26, 2019No Comments

Initial User Research

To understand the users of the Noun Project or similar icon search engines (aggregators), I came up with few questions and scenarios. Filtering out those who have experience with using an icon aggregator resulted a group of people who are in art/design industry. With the selected people, I proceeded the following questions and scenarios.


Q. When you’re using the Noun Project or similar icon search engines, how many icons are you mostly looking for?

  • 82% answered they often look for more than a single icon
  • 18% answered they look for just one icon

Q. If you are looking for multiple icons, do you prefer them to be created by one person/designer?

  • 91% answered (Y) they prefer their icons to be created by one person/designer
  • 9% answered (N) they don't particularly prefer icons to be created by one person/designer

Q. Please briefly describe why you picked Y/N for the previous question.

  • Y: consistency, same/similar style, overall accordance, uniformity, common theme, "already have a specific icon in my mind"
  • N: reference, "not for the right-away use"


S. You are looking for 2 icons: a dog and a cat. You found a dog icon that you like, but the creator’s collection only has icons of a dog. What would be your action?

three icons of dogs in similar style and one question mark
  • modify one of the icons from the collection so that it can look a bit like a cat
  • I'd look for a cat icon from someone else that looks as similar as the dog icon I found
  • I'd find a collection that has both dog and cat
  • Give up the cat (if i have the budget, recruit a cat icon designer) ...

S. You are designing a website, and need 2 types of icons: a cake and a cat. Do you think this set of icons are good to be used together? Why?

a cake icon and a cat icon with similar style
  • yes, they look same in style. Maybe the cat is too big?
  • Yes. The two icons have used same background color and a line stroke.
  • Yes, both in simple lines and fluffy.
  • No, the porportion could be better (the cake smaller than the cat or the cat as face only) ...

S. You are designing an app, and need 4 types of icons: flower, bicycle, cake and dog. Do you think this set of icons are good to be used together? Why?

flower, bicycle, cake and dog icons. bicycle icon has different weight
  • No, the bicycle icon is not designed like the others
  • the bicycle looks too random with other ones
  • No. The bike icon's line stroke is too thin compare to the other three.
  • No, that bicycle looks off (less noticeable) with other icons ...


  1. have more realistic/practical set of icons (such as doc, image, folder..etc that are often seen or will be used in product design) so the user scenarios can be smoothly understood
  2. make sure this is not about validating icon’s meaning
  3. focus on the main users of the Noun Project or of similar services (not the viewers of their final products)
  4. main topic moving away from perception of pictograms, but more towards expanding the possibility of collaboration inside an icon aggregator and improve its search experience

February 26, 2019No Comments

VGG16 & Keras

Using a pre-trained model in Keras, e.g., VGG, to extract the feature of a given image;

February 19, 2019No Comments

Context/Research Summary

Influential precursor projects, products, research, installations, performances and/or other influencers within which my thesis project sits.

  • Describe each project specifically, include images
  • Explain how your thesis project has been inspired/informed by each 
  • Explain how your project will build on each 
  • Explain how your project will be different from each

The Noun Project

“Noun Project.” The Noun Project,

the Noun Project screenshot

The Noun Project is a website that aggregates and catalogs pictograms created and uploaded by graphic designers around the world, with the largest dataset. My project will base on its API, because I found their library most diverse yet has simplest forms. The website uses search by tag, and that is something I'll try to make difference. Link to related thread:

Modern Pictograms for Lottie

“Modern Pictograms for Lottie.” Airbnb Design,

Gif from Modern Pictograms for Lottie

Salih Abdul-Karim, a motion designer at Airbnb, experiments various ways to create animation friendly icons -- or “artworks”, according to his description. He gives an idea what are the basic components of an icon, and further, what does the Noun Project's community aims to be, which is a collaboration between different people and overlapping practices. My project will be irrelevant from animating icon, but the article is inspiring in a way that it brings up the nature of the Noun Project's community.

Brandmark Logo Maker

“Brandmark - the Smart Logo Maker.” Brandmark Logo Maker - the Most Advanced AI Logo Design Tool,

Brandmark - the Smart Logo Maker screen shot

Brandmark uses deep learning tools to generate logos composed of an icon, typography and color scheme. It uses a convolutional net to filter out common symbols and shapes that are not "brandable", by giving legibility score and uniqueness score. Technically, it's exactly what I attempt to achieve -- except that I have different approach on sorting icons, as I'm not interested in making a single icon standing out as a unique logo.

February 19, 2019No Comments

Production Schedule/Implementation Plan

Until Feb 15concept development, brief technical testing research
Feb 16 ~ March 5extract visual patterns from the Noun Project, user test of the 'audiences' who view iconsresearch, wireframes, user flow, coding
March 6 ~ April 16visualize the analysis, start designing the prototype of interface based on the previous results, user testing of the 'producers' who search and include icons in their workhigh-fidelity prototype, usability test, coding
April 17 ~ May 6finish prototype design and make presentationhigh-fidelity prototype, presentation prep

February 15, 2019No Comments

Consistent Doodles

  1. A pictogram, also called an icon, is an ideogram that conveys its meaning through its pictorial resemblance. For my thesis, I would like to design a search engine that effectively organizes pictograms by their visual similarities. Its goal is to help both viewers and producers of a visual system by maintaining consistency in pictograms.
  2. Pictograms vary in their level of details, scale, weight, and style depending on their themes and purposes. However, at least inside one system (i.e. a mobile application), they should align together; else they become mere doodles than a language by lack of consistency. It's a special subject to me, because I've been studying and working as a graphic designer and seen many users or designers getting confused about the subject. Providing clear way to categorize icons not only benefits the workflow of designers/producers, but also helps the viewers of that designed output, such as a presentation or a website.
  3. The final output will be a prototype of a web based search engine involving the Noun Project API, which has the largest icon database.


  1. My thesis is heavily influenced by machine learning projects as Font Map or Quick, Draw! -- which both build relationship between visual elements by analyzing their patterns. Applying deep learning techniques to icons also has been done before (, but only for the purpose of "logo maker". It's great that I have many similar projects, so my research started from studying them.
  2. After studying the existing examples, I've learned that it is possible to sort pictograms. To start, I needed a large source of icons and I decided to use the Noun Project API, attracted by its high number of collection. Hence, my pictogram searching engine will be based of the Noun Project. The deep learning process is not only for designing the final search tool, but also to not arbitrarily pick icons for my research and user testing.
  3. During the process, I've already started conversation with some graphic designers about how they feel about the current method of searching icons. I'm also taking courses from Gene Kogan and Yining Shi, who are both experts in machine learning -- so I'll reach out to them for technical advice.
  4. George, Alison. “Code Hidden in Stone Age Art May Be the Root of Human Writing.” New Scientist, New Scientist, 2016,
    Ho, Kevin. “Organizing the World of Fonts with AI – IDEO Stories – Medium.”, Medium, 20 Apr. 2017,
    Drucker, Johanna. The Visible Word: Experimental Typography and Modern Art. University of Chicago Press, 1997.
    “Branded in Memory.”
    “Brandmark - the Smart Logo Maker.” Brandmark Logo Maker - the Most Advanced AI Logo Design Tool,
  5. Schedule:
    • Until Feb 15: concept development, brief technical testing
    • Feb 16 ~ March 5: extract visual patterns from the Noun Project, user test of the 'audiences' who view icons
    • March 6 ~ April 16: visualize the analysis, start designing the prototype of interface based on the previous results, user testing of the 'producers' who search and include icons in their work
    • April 17 ~ May 6: finish prototype design and make presentation
  6. The part that still needs to be found out is the portion of actual programming in prototype. While the final prototype will be built with Sketch toolkit to present my idea as clear as possible, its basic interface will be based of some working code. This is something that will be continuously adjusted throughout the research.


  1. I have relatively little amount of knowledge about machine learning, and it's the core part for my research. To improve my understanding, I'm taking two other ML related courses. It's not that I'm trying to build a whole new type of convolutional net, and as I mentioned, there are many similar examples that already exist. My thesis is more centered around how to apply the existing technology to improve the Noun Project service, than inventing a new one. In case of technical difficulties, I'm planning to ask for advice from Gene and Yining, while I'm already gaining a lot of resources from Gene's course.
  2. My thesis is composed of two big parts of analysis and prototype. The analysis part is pretty straightforward: I extract visual similarities from the Noun Project icons, and test if the viewers have easier time with the icon set in consistency, compared to the one without it. There are bit more unknowns in the prototype part, regarding how much programming will be actually done as a final output.
  3. The analysis will be visualized with at least two types of different icons, thus it can show the relationship between multiple terms. The prototype lets the users to pick multiple terms and get a set of icons with similarity, or change them altogether simultaneously.

February 11, 2019No Comments

Concept Development 2

This week was much about narrowing down my topic and looking for available implements. During last week's group meeting, I proposed that my focus will lean towards pictogram for following reasons:

  1. Alphabets vary enormously in characteristics such as their typography and structure. I've done a project about my bilingualism, and the biggest challenge for expanding the project was my ignorance about other types of alphabets. Even a single alphabet has its own complicated system (often deeply connected to its culture), and I would like approach my thesis in the point of visual perception, rather than focusing on a specific alphabet or a culture.
  2. Similar from the first reason, I would like to pick a language that is more universal and primitive.


"A pictogram, also called a pictogramme, pictograph, or simply picto, and in computer usage an icon, is an ideogram that conveys its meaning through its pictorial resemblance to a physical object. Pictographs are often used in writing and graphic systems in which the characters are to a considerable extent pictorial in appearance."

Google Material Design Icons
Google Material Design Icons

Doodles, Pictograms, and Letters

Language is the system of signs. It is “a storehouse filled by the members of a given community through their active use of speaking, a grammatical system that has a potential existence in each brain, or, more specifically, in the brains of a group of individuals" – Ferdinand de Saussure 13-14 in Vidra-Mitra, 2017

Cave paintings in Magura Cave, Bulgaria. Photo by
Magura Cave, Bulgaria. Photo by

A pictogram is somewhere between a primitive drawing and a letter; not only in their form, but also in their historical order. It's more systematic than a doodle, yet requires less abstraction and training than a letter. It's is a doodle, yet a "consistent doodle".

ISO 7001 (public information symbols)

"ISO 7001 ('public information symbols') is a standard published by the International Organization for Standardization that defines a set of pictograms and symbols for public information."

Japanese green exit sign with Running Man moving to the left through a doorway

For example, the international exit sign, "Running Man", was introduced into the 1987 standard as a consistent and international approach to move away from using words in the native language, after the Sennichi Department Store Building fire.

Consistency and System

Ferdinand de Saussure claims language is relational system of signs. One of the problems about a pictogram is that it's much diverse in style and interpretation of what it signifies.

two cat icons from the noun project

These two icons are both from searching "cat" in the Noun Project. The Noun Project is the biggest platform that shares such pictograms, created and uploaded by graphic designers around the world.

A pictogram can be a "doodle" than a "language", by lack of consistency.. Although applying regulation with a single standard as ISO 7001 is the simplest answer, it's not always an available option -- especially regarding that a pictogram is ultimately another form of creative expression. Also, they're rather recommended to be different in their level of details, scale or weight, depending on their purposes and environments. However, at least inside one system (i.e. a mobile application), they have to align together; else they become mere doodles.

When people open a website or enter into a building, they're entering into a new system. Something that can improve and build the legibility of pictograms is the consistency within a system, instead of the consistency over all the systems (like ISO 7001). It's much more achievable and friendly approach for keeping pictograms as a language, without destroying their diversity. It will also give a nice observation of how people visually perceive things and interpret them.


  • The Noun Project API will be the source of my thesis
  • Items should be collected as /icons/{term} than /icons/{collection} (already formed collection)
  • maximum number limit of call is 50? (reach out to the Noun Project)
  • let's not use creative commons, but public works only
  • Possibilities of application: better way of sorting icons compared to tag-base categorization, draw and search icon, change icons altogether in similar style

February 5, 2019No Comments

Concept Development

With "this", world's every graphic asset builds connection, instead of standing in its own island. As it focuses on simplified 2D imagery, its accuracy can be improved by catching an image with one set of color contrast--rather than complex objects or scenery. It doesn't necessarily try to translate the real world, but it observes how people perceive and recreate it in their own system. –Dream review

Biltmore Ave by Mason Adams


Concept: Graphic elements, such as icons and signages are everywhere in people's daily life. They also have standards and people have no problem communicating via those elements, or they do have problems. Regardless, the ability to produce a system of signs is human's very own characteristic.

Font Map by Kevin Ho: Also continues to, which is a website you can pair font based on font vectors. It touches the core of my idea in a sense that the project itself tries to defend its own decision making in graphic design. My improvement or difference can possibly rely on ways to present this kind of analysis into other formats or applications.

Audience: There can be two types of users. One type can be those who are vaguely interested in visual communications area, and want to learn more about it. The other one can be those who desire to take these visual languages in different perspective.

Context: Due to the nature of my topic, it's somewhere between visual design and data analysis. There are many cases that attempt those two combinations, such as Programming Design Systems or Color of words. Although area of graphic design hasn't been fully observed, there are plenty of studies about road-sign recognition due to technical improvement in vehicle AI.

Content: There are still parts to be solidified, such as a method to tightly bond the daily experience of encountering numerous graphic elements with my analysis. Besides, I think the analysis itself will be consisted of big two themes of form and color. There is also a possibility to go with only one of them, depending on my speed.

Tech: Building T-SNE vector space and analyzing data have been done before, yet I haven't implemented in more advanced way than mere data files. In the same line from building content, I would like this to be approached in casual way as well.

Timeframe & Scope: 14 Weeks will be enough.


Preliminary Thesis Statement

Nowadays, people use Google Map navigation to find their direction. Before (or still), it was all about paper maps and road symbols. Even before that, people had similar system, drawn on cave walls and trees. Although now most things are migrating into digital devices, visual communication always exist in human history, only within different medium.

Visual communication, or graphic design, is like culture and language. It evolved in unpredictable directions, and people's lifestyle and history are embedded in them. Therefore, it's rather a new way to attempt these visual elements in analytical point of view, solely based on their appearance without their background contents. For the same reason, the concept or prototype I'll be building won't be about judging good or bad designs.

Instead, it's more about carefully observing what surround us everyday. My final outcome will be a combination of analysis and a tool to detect and/or generate visual elements based on that analysis.

December 12, 2018No Comments

Ghosts and Words

running example of Ghosts and words: pac-mans represented by yellow number eats food, or encounter ghost names and die

Ghost and words is a continuation from Evolutionary Pac-Man. It uses the same strategy of breaking down words (in this case, using paragraphs from Wikipedia's Pac-Man page) as "food", and also having "poison" or "ghost" that can kill the pac-mans.

Didn't know those ghosts have names...

The initial idea was simple - I wanted the words to act and animate as vectors. It also seemed interesting to use the correlation between actual meanings of words through such method like Word2vec, but I got stuck and spent too much time on other parts and didn't really get to that level.

Evolutionary Pac-Man was revisited because it was already using one unique way of converting text into many vector points. The renewed version more specifically focuses on typographic aspects, by turning (almost) every element into text.

In this game, texts do many things. They get eaten, run away or run towards, or sometimes get killed or starve to death.


The "ghosts" and "food" were initially built as classes:

class Ghost {
    this.x = x;
    this.y = y;
    this.type = ["BLINKY","PINKY","INKY","CLYDE"];
    this.color = ["red", "pink", "cyan", "orange"]; = int(random(4))

    textAlign(CENTER, CENTER);
    text(this.type[], this.x + random(3), this.y + random(3));

However, as I start inserting pac-man object, I realized in order to use p5.Vector functions such as p5.Vector.dist, I had to keep them as p5 Vectors:

    for (let i = list.length - 1; i >= 0; i--) {
      // Calculate distance
      let d = p5.Vector.dist(list[i], this.position);
      // If it's within perception radius and closer than pervious
      if (d < this.dna[2 + index] && d < closestD) {
        closestD = d;
        // Save it
        closest = list[i];

So at this point, it's ran with four different arrays of ghost, which I think is bit redundant and I would like to make it simpler in the future.