Consistent Doodles (2019)

Icon search engine design with machine learning, using pre-trained feature extraction

"A pictogram, also called a pictogramme, pictograph, or simply picto, and in computer usage an icon, is an ideogram that conveys its meaning through its pictorial resemblance to a physical object. Pictographs are often used in writing and graphic systems in which the characters are to a considerable extent pictorial in appearance."

google-material-design-icons
Google Material Design Icons
cave_painting-300×225-1
Magura Cave, Bulgaria. Photo by flickr.com 

Doodles, Pictograms, and Letters

Language is the system of signs. It is “a storehouse filled by the members of a given community through their active use of speaking, a grammatical system that has a potential existence in each brain, or, more specifically, in the brains of a group of individuals" – Ferdinand de Saussure 13-14 in Vidra-Mitra, 2017

A pictogram is somewhere between a primitive drawing and a letter; not only in their form, but also in their historical order. It's more systematic than a doodle, yet requires less abstraction and training than a letter. It's is a doodle, yet a "consistent doodle".

universal-design-meets-the-exit-sign-japanese-exit-sign-with-running-man

For example, "ISO 7001 ('public information symbols') is a standard published by the International Organization for Standardization that defines a set of pictograms and symbols for public information."

The international exit sign, "Running Man", was introduced into the 1987 standard as a consistent and international approach to move away from using words in the native language, after the Sennichi Department Store Building fire.

Consistency and System

Ferdinand de Saussure claims language is relational system of signs. One of the problems about a pictogram is that it's much diverse in style and interpretation of what it signifies.

thesis-cats

These two icons are both from searching "cat" in the Noun Project. The Noun Project is the biggest platform that shares such pictograms, created and uploaded by graphic designers around the world.

A pictogram can be a "doodle" than a "language", by lack of consistency.. Although applying regulation with a single standard as ISO 7001 is the simplest answer, it's not always an available option -- especially regarding that a pictogram is ultimately another form of creative expression. Also, they're rather recommended to be different in their level of details, scale or weight, depending on their purposes and environments. However, at least inside one system (i.e. a mobile application), they have to align together; else they become mere doodles.

When people open a website or enter into a building, they're entering into a new system. Something that can improve and build the legibility of pictograms is the consistency within a system, instead of the consistency over all the systems (like ISO 7001). It's much more achievable and friendly approach for keeping pictograms as a language, without destroying their diversity. It will also give a nice observation of how people visually perceive things and interpret them.

The Noun Project

The Noun Project is a website that aggregates and catalogs pictograms created and uploaded by graphic designers around the world, with the largest dataset. I used the Noun Project API to explore beyond its tag-based categorization.

thesis-dogs
thesis-dogs2

Above images are t-SNE visualization of using a pre-trained model in Keras, i.e. VGG, to extract the feature gvien images. These were the helpful links to the above process:

Some other personal notes:

  • The dissimilarity metric I used is cosine distance ↗ (do the vectors point in the same direction?). Also considered:
  • alpha_failed.ipynb ↗ Solve the alpha channel issue for feature extraction; directly pull images from the API
consistent-survey

Besides the technical studies, I came up with few questions to understand the users of the Noun Project or similar icon search engines (aggregators). I narrowed down the participants to those who have experience with using an icon aggregator, resulting a group of people who are in art/design industry. With the selected group, I proceeded the following questions and scenarios.

Q. When you’re using the Noun Project or similar icon search engines, how many icons are you mostly looking for?

  • 82% answered they often look for more than a single icon
  • 18% answered they look for just one icon

Q. If you are looking for multiple icons, do you prefer them to be created by one person/designer?

  • 91% answered (Y) they prefer their icons to be created by one person/designer
  • 9% answered (N) they don't particularly prefer icons to be created by one person/designer

Q. Please briefly describe why you picked Y/N for the previous question.

  • Y: consistency, same/similar style, overall accordance, uniformity, common theme, "already have a specific icon in my mind"
  • N: reference, "not for the right-away use"

Q. You are looking for 2 icons: a dog and a cat. You found a dog icon that you like, but the creator’s collection only has icons of a dog. What would be your action?

testing-thesis- 3
  • "modify one of the icons from the collection so that it can look a bit like a cat"
  • "I'd look for a cat icon from someone else that looks as similar as the dog icon I found"
  • "I'd find a collection that has both dog and cat"
  • "Give up the cat (if i have the budget, recruit a cat icon designer)..."

Q. You are designing a website, and need 2 types of icons: a cake and a cat. Do you think this set of icons are good to be used together? Why?

testing-thesis- 1 copy
  • "yes, they look same in style. Maybe the cat is too big?"
  • "Yes. The two icons have used same background color and a line stroke."
  • "Yes, both in simple lines and fluffy."
  • "No, the porportion could be better (the cake smaller than the cat or the cat as face only)..."

Q. You are designing an app, and need 4 types of icons: flower, bicycle, cake and dog. Do you think this set of icons are good to be used together? Why?

testing-thesis- 1
  • "No, the bicycle icon is not designed like the others"
  • "the bicycle looks too random with other ones"
  • "No. The bike icon's line stroke is too thin compare to the other three."
  • "No, that bicycle looks off (less noticeable) with other icons..."

Feature Extraction for Icon Searching Experience

The functionality of feature extraction suited well with the nature of pictogram, where the visual consistency is essential. Based on the above exploration, I proposed few ideas to improve icon searching experience in the Noun Project – which is applicable to any type of icon aggregator model.

consistent-iconAggregator
ConsistentDoodles_userFlow3
consistent-tech

In a summary, there are two specific features I proposed:

  • Related Recommendation for searching single icon:
    • Feeds visually similar icons, once the user makes an initial selection
  • Set Recommendation for searching multiple icon:
    • Matches visually similar icons with different text queries
consistent-match2
consistent-search

The project was presented as my 2019 thesis ↗ for Interactive Telecommunications Program at New York University.

Back to top Arrow