Solve the alpha channel issue for feature extraction; directly pull images from the API
"This is due to the fact that eps does not know about transparency and the default background color for the rasterization was (0, 0, 0, 0) (black which is fully transparent)." --- continuing error in analysis + expanding features
def load_image(path): img = Image.open(path) img = scipy.misc.imresize(np.array(img), (224, 224), interp='bicubic') img[:,:,0] = 255.0-img[:,:,3] img[:,:,1] = 255.0-img[:,:,3] img[:,:,2] = 255.0-img[:,:,3] img = img[:,:,:3] x = image.img_to_array(img) x = np.expand_dims(x, axis=0) x = preprocess_input(x) return img, x
To understand the users of the Noun Project or similar icon search engines (aggregators), I came up with few questions and scenarios. Filtering out those who have experience with using an icon aggregator resulted a group of people who are in art/design industry. With the selected people, I proceeded the following questions and scenarios.
Questions
Q. When you’re using the Noun Project or similar icon search engines, how many icons are you mostly looking for?
82% answered they often look for more than a single icon
18% answered they look for just one icon
Q. If you are looking for multiple icons, do you prefer them to be created by one person/designer?
91% answered (Y) they prefer their icons to be created by one person/designer
9% answered (N) they don't particularly prefer icons to be created by one person/designer
Q. Please briefly describe why you picked Y/N for the previous question.
Y: consistency, same/similar style, overall accordance, uniformity, common theme, "already have a specific icon in my mind"
N: reference, "not for the right-away use"
Scenarios
S. You are looking for 2 icons: a dog and a cat. You found a dog icon that you like, but the creator’s collection only has icons of a dog. What would be your action?
modify one of the icons from the collection so that it can look a bit like a cat
I'd look for a cat icon from someone else that looks as similar as the dog icon I found
I'd find a collection that has both dog and cat
Give up the cat (if i have the budget, recruit a cat icon designer) ...
S. You are designing a website, and need 2 types of icons: a cake and a cat. Do you think this set of icons are good to be used together? Why?
yes, they look same in style. Maybe the cat is too big?
Yes. The two icons have used same background color and a line stroke.
Yes, both in simple lines and fluffy.
No, the porportion could be better (the cake smaller than the cat or the cat as face only) ...
S. You are designing an app, and need 4 types of icons: flower, bicycle, cake and dog. Do you think this set of icons are good to be used together? Why?
No, the bicycle icon is not designed like the others
the bicycle looks too random with other ones
No. The bike icon's line stroke is too thin compare to the other three.
No, that bicycle looks off (less noticeable) with other icons ...
Learning/Conclusion
have more realistic/practical set of icons (such as doc, image, folder..etc that are often seen or will be used in product design) so the user scenarios can be smoothly understood
make sure this is not about validating icon’s meaning
focus on the main users of the Noun Project or of similar services (not the viewers of their final products)
main topic moving away from perception of pictograms, but more towards expanding the possibility of collaboration inside an icon aggregator and improve its search experience
Influential precursor projects, products, research, installations, performances and/or other influencers within which my thesis project sits.
Describe each project specifically, include images
Explain how your thesis project has been inspired/informed by each
Explain how your project will build on each
Explain how your project will be different from each
The Noun Project
“Noun Project.” The Noun Project, thenounproject.com/.
The Noun Project is a website that aggregates and catalogs pictograms created and uploaded by graphic designers around the world, with the largest dataset. My project will base on its API, because I found their library most diverse yet has simplest forms. The website uses search by tag, and that is something I'll try to make difference. Link to related thread: https://www.quora.com/What-is-the-best-icon-library
Modern Pictograms for Lottie
“Modern Pictograms for Lottie.” Airbnb Design, airbnb.design/modern-pictograms-for-lottie/.
Salih Abdul-Karim, a motion designer at Airbnb, experiments various ways to create animation friendly icons -- or “artworks”, according to his description. He gives an idea what are the basic components of an icon, and further, what does the Noun Project's community aims to be, which is a collaboration between different people and overlapping practices. My project will be irrelevant from animating icon, but the article is inspiring in a way that it brings up the nature of the Noun Project's community.
Brandmark Logo Maker
“Brandmark - the Smart Logo Maker.” Brandmark Logo Maker - the Most Advanced AI Logo Design Tool, brandmark.io/intro/.
Brandmark uses deep learning tools to generate logos composed of an icon, typography and color scheme. It uses a convolutional net to filter out common symbols and shapes that are not "brandable", by giving legibility score and uniqueness score. Technically, it's exactly what I attempt to achieve -- except that I have different approach on sorting icons, as I'm not interested in making a single icon standing out as a unique logo.
extract visual patterns from the Noun Project, user test of the 'audiences' who view icons
research, wireframes, user flow, coding
March 6 ~ April 16
visualize the analysis, start designing the prototype of interface based on the previous results, user testing of the 'producers' who search and include icons in their work
A pictogram, also called an icon, is an ideogram that conveys its meaning through its pictorial resemblance. For my thesis, I would like to design a search engine that effectively organizes pictograms by their visual similarities. Its goal is to help both viewers and producers of a visual system by maintaining consistency in pictograms.
Pictograms vary in their level of details, scale, weight, and style depending on their themes and purposes. However, at least inside one system (i.e. a mobile application), they should align together; else they become mere doodles than a language by lack of consistency. It's a special subject to me, because I've been studying and working as a graphic designer and seen many users or designers getting confused about the subject. Providing clear way to categorize icons not only benefits the workflow of designers/producers, but also helps the viewers of that designed output, such as a presentation or a website.
The final output will be a prototype of a web based search engine involving the Noun Project API, which has the largest icon database.
Research
My thesis is heavily influenced by machine learning projects as Font Map or Quick, Draw! -- which both build relationship between visual elements by analyzing their patterns. Applying deep learning techniques to icons also has been done before (brandmark.io/intro/), but only for the purpose of "logo maker". It's great that I have many similar projects, so my research started from studying them.
After studying the existing examples, I've learned that it is possible to sort pictograms. To start, I needed a large source of icons and I decided to use the Noun Project API, attracted by its high number of collection. Hence, my pictogram searching engine will be based of the Noun Project. The deep learning process is not only for designing the final search tool, but also to not arbitrarily pick icons for my research and user testing.
During the process, I've already started conversation with some graphic designers about how they feel about the current method of searching icons. I'm also taking courses from Gene Kogan and Yining Shi, who are both experts in machine learning -- so I'll reach out to them for technical advice.
Until Feb 15: concept development, brief technical testing
Feb 16 ~ March 5: extract visual patterns from the Noun Project, user test of the 'audiences' who view icons
March 6 ~ April 16: visualize the analysis, start designing the prototype of interface based on the previous results, user testing of the 'producers' who search and include icons in their work
April 17 ~ May 6: finish prototype design and make presentation
The part that still needs to be found out is the portion of actual programming in prototype. While the final prototype will be built with Sketch toolkit to present my idea as clear as possible, its basic interface will be based of some working code. This is something that will be continuously adjusted throughout the research.
Implementation
I have relatively little amount of knowledge about machine learning, and it's the core part for my research. To improve my understanding, I'm taking two other ML related courses. It's not that I'm trying to build a whole new type of convolutional net, and as I mentioned, there are many similar examples that already exist. My thesis is more centered around how to apply the existing technology to improve the Noun Project service, than inventing a new one. In case of technical difficulties, I'm planning to ask for advice from Gene and Yining, while I'm already gaining a lot of resources from Gene's course.
My thesis is composed of two big parts of analysis and prototype. The analysis part is pretty straightforward: I extract visual similarities from the Noun Project icons, and test if the viewers have easier time with the icon set in consistency, compared to the one without it. There are bit more unknowns in the prototype part, regarding how much programming will be actually done as a final output.
The analysis will be visualized with at least two types of different icons, thus it can show the relationship between multiple terms. The prototype lets the users to pick multiple terms and get a set of icons with similarity, or change them altogether simultaneously.
This week was much about narrowing down my topic and looking for available implements. During last week's group meeting, I proposed that my focus will lean towards pictogram for following reasons:
Alphabets vary enormously in characteristics such as their typography and structure. I've done a project about my bilingualism, and the biggest challenge for expanding the project was my ignorance about other types of alphabets. Even a single alphabet has its own complicated system (often deeply connected to its culture), and I would like approach my thesis in the point of visual perception, rather than focusing on a specific alphabet or a culture.
Similar from the first reason, I would like to pick a language that is more universal and primitive.
Pictogram
"A pictogram, also called a pictogramme, pictograph, or simply picto, and in computer usage an icon, is an ideogram that conveys its meaning through its pictorial resemblance to a physical object. Pictographs are often used in writing and graphic systems in which the characters are to a considerable extent pictorial in appearance."
Google Material Design Icons
Doodles, Pictograms, and Letters
Language is the system of signs. It is “a storehouse filled by the members of a given community through their active use of speaking, a grammatical system that has a potential existence in each brain, or, more specifically, in the brains of a group of individuals" – Ferdinand de Saussure 13-14 in Vidra-Mitra, 2017
Magura Cave, Bulgaria. Photo by flickr.com
A pictogram is somewhere between a primitive drawing and a letter; not only in their form, but also in their historical order. It's more systematic than a doodle, yet requires less abstraction and training than a letter. It's is a doodle, yet a "consistent doodle".
ISO 7001 (public information symbols)
"ISO 7001 ('public information symbols') is a standard published by the International Organization for Standardization that defines a set of pictograms and symbols for public information."
For example, the international exit sign, "Running Man", was introduced into the 1987 standard as a consistent and international approach to move away from using words in the native language, after the Sennichi Department Store Building fire.
Consistency and System
Ferdinand de Saussure claims language is relational system of signs. One of the problems about a pictogram is that it's much diverse in style and interpretation of what it signifies.
These two icons are both from searching "cat" in the Noun Project. The Noun Project is the biggest platform that shares such pictograms, created and uploaded by graphic designers around the world.
A pictogram can be a "doodle" than a "language", by lack of consistency.. Although applying regulation with a single standard as ISO 7001 is the simplest answer, it's not always an available option -- especially regarding that a pictogram is ultimately another form of creative expression. Also, they're rather recommended to be different in their level of details, scale or weight, depending on their purposes and environments. However, at least inside one system (i.e. a mobile application), they have to align together; else they become mere doodles.
When people open a website or enter into a building, they're entering into a new system. Something that can improve and build the legibility of pictograms is the consistency within a system, instead of the consistency over all the systems (like ISO 7001). It's much more achievable and friendly approach for keeping pictograms as a language, without destroying their diversity. It will also give a nice observation of how people visually perceive things and interpret them.
Notes
The Noun Project API will be the source of my thesis
Items should be collected as /icons/{term} than /icons/{collection} (already formed collection)
maximum number limit of call is 50? (reach out to the Noun Project)
let's not use creative commons, but public works only
Possibilities of application: better way of sorting icons compared to tag-base categorization, draw and search icon, change icons altogether in similar style
With "this", world's every graphic asset builds connection, instead of standing in its own island. As it focuses on simplified 2D imagery, its accuracy can be improved by catching an image with one set of color contrast--rather than complex objects or scenery. It doesn't necessarily try to translate the real world, but it observes how people perceive and recreate it in their own system. –Dream review
Biltmore Ave by Mason Adams
Research
Concept: Graphic elements, such as icons and signages are everywhere in people's daily life. They also have standards and people have no problem communicating via those elements, or they do have problems. Regardless, the ability to produce a system of signs is human's very own characteristic.
Font Map by Kevin Ho: Also continues to fontjoy.com, which is a website you can pair font based on font vectors. It touches the core of my idea in a sense that the project itself tries to defend its own decision making in graphic design. My improvement or difference can possibly rely on ways to present this kind of analysis into other formats or applications.
Audience: There can be two types of users. One type can be those who are vaguely interested in visual communications area, and want to learn more about it. The other one can be those who desire to take these visual languages in different perspective.
Context: Due to the nature of my topic, it's somewhere between visual design and data analysis. There are many cases that attempt those two combinations, such as Programming Design Systems or Color of words. Although area of graphic design hasn't been fully observed, there are plenty of studies about road-sign recognition due to technical improvement in vehicle AI.
Content: There are still parts to be solidified, such as a method to tightly bond the daily experience of encountering numerous graphic elements with my analysis. Besides, I think the analysis itself will be consisted of big two themes of form and color. There is also a possibility to go with only one of them, depending on my speed.
Tech: Building T-SNE vector space and analyzing data have been done before, yet I haven't implemented in more advanced way than mere data files. In the same line from building content, I would like this to be approached in casual way as well.
Nowadays, people use Google Map navigation to find their direction. Before (or still), it was all about paper maps and road symbols. Even before that, people had similar system, drawn on cave walls and trees. Although now most things are migrating into digital devices, visual communication always exist in human history, only within different medium.
Visual communication, or graphic design, is like culture and language. It evolved in unpredictable directions, and people's lifestyle and history are embedded in them. Therefore, it's rather a new way to attempt these visual elements in analytical point of view, solely based on their appearance without their background contents. For the same reason, the concept or prototype I'll be building won't be about judging good or bad designs.
Instead, it's more about carefully observing what surround us everyday. My final outcome will be a combination of analysis and a tool to detect and/or generate visual elements based on that analysis.
Ghost and words is a continuation from Evolutionary Pac-Man. It uses the same strategy of breaking down words (in this case, using paragraphs from Wikipedia's Pac-Man page) as "food", and also having "poison" or "ghost" that can kill the pac-mans.
Full Screen (interact with left & right arrow keys + clicking)
The initial idea was simple - I wanted the words to act and animate as vectors. It also seemed interesting to use the correlation between actual meanings of words through such method like Word2vec, but I got stuck and spent too much time on other parts and didn't really get to that level.
Evolutionary Pac-Man was revisited because it was already using one unique way of converting text into many vector points. The renewed version more specifically focuses on typographic aspects, by turning (almost) every element into text.
In this game, texts do many things. They get eaten, run away or run towards, or sometimes get killed or starve to death.
process
The "ghosts" and "food" were initially built as classes:
However, as I start inserting pac-man object, I realized in order to use p5.Vector functions such as p5.Vector.dist, I had to keep them as p5 Vectors:
for (let i = list.length - 1; i >= 0; i--) {
// Calculate distance
let d = p5.Vector.dist(list[i], this.position);
// If it's within perception radius and closer than pervious
if (d < this.dna[2 + index] && d < closestD) {
closestD = d;
// Save it
closest = list[i];
So at this point, it's ran with four different arrays of ghost, which I think is bit redundant and I would like to make it simpler in the future.
