Jianhao Ma's IMA Blog

 Reading Response and Markov Chain

 A6: Quick Draw! About Quick Draw Data Set Quick Draw is a representative machine learning & data collection project, it shows an ideal design model for public machine learning and data set projects.  It's simply designed to be engaging and accessible. And such accessibility profligates the data collection progress. It uses the classification model trained with collected data as a feedback to reward participants.  Although there are several potential problems with the dataset such as latent bias, I still found it very helpful for beginners. Both the doodle and the label are small objects (compared to other visual machine learning projects), therefore are easy and fast to process. It makes ambitious attempts possible for machine learning learners.  DoodleNet p5 Sketch https://editor.p5js.org/roger1mjh/sketches/kPVq2Prxl For this week's homework, I worked on the "two canvases" template. My idea is to visually represent and store the "labels", "confide...

Exploring Datasets and Practice on 311 World Happiness Report I've seen other datasets from Gallup World Poll before and was interested in how it quantifies happiness and its parameters. Here's the details of this dataset:     Collector/contributor:   United Nations     Purpose : Happiness Indicators to inform government's decision makings     Collection Method:  Data are collected from Gallup World Poll     Dimensions:  156 lines(countries), 12 columns The dataset quantifies happiness of citizens from different countries from a macro perspectives. It combines individual polls and objective statistics. It's convenient to use machine learning and do a regression to estimate the correlation between Happiness and each parameters(GDP, life expectancy, etc..) 311 p5 sketch exercise https://editor.p5js.org/roger1mjh/sketches/ZG2BkJS07 Originally I want to make some edits on the existed code to conduct a different use of the simplifi...

 Image Transfer on P5 Response to Dr. Fiebrink's Speech: There's one part in this speech that intrigues me the most, which is using Machine Learning in creative works does not need to be smart and delicate. It is OK to be dumb and funny, because that's what an unmatured technology genuinely looks like, and it can still be creative. The Edges to Cat example is so fun to play with that I'm kind of addict to it.  Attempt to do Image Transfer on P5 with Pose Regression I watched the videos posted and was most interested in the pose regression example. But the RGB value shift looks not "machine learningly" enough, because it's possible to make some similar project without neural network. Therefore I tried to make one with image transfer. Firstly, I need to load image in data format in P5. I decided to try pixels first, because they retrieve and store image data directly. Then I console logged the "pixels" object in draw, to test out how much the data ...

 A3: Applications of Machine Learning models Jianhao Ma Response to PoseNet's Data & Model Biography To be more specific the biography provided is for MobileNet. However, I did learn something about PoseNet because it's the sub model of MobileNet.  One important thing I noticed is that the model is optimized for devices with limited performance, ones that can be easily embedded and require less power. Therefore, it can be applied to outdoor activities in daily life scenarios. Moreover, PoseNet does a really good job in inspecting and classifying people's poses and gestures. All of the followings inspired me to make my project this week. Have A Safe Ride My intention is to make a demo of an application that detects motorcycle hand signals.  My plan for the application is to embed them in auto-driving and safety technologies on automobiles. It enables cars to recognize the hand signals of bikers in front of them and responses safely based on that. I built my project on ...

2A: Excavating AI, Teachable Machine Project Jianhao Ma Moral Machines The reading "Excavating AI" brings up an idea that the tasks of training AI should not just be making them smarter, but to apply political consideration and moral standard as well, especially when related to people. I would not care about such idea years ago, because I thought that technical breakthroughs is the one and foremost priority in emerging technology industries, why bother? However, when the application of machine learning started entering our lives, I changed my mind. Everyday, there are many information processed/filtered by machine learning tools being fed to us, textual or non textual. They could be the advertisement you watch on social media, the pop-up news on your phone. In other words, many of the lower products of machine learning are mediums. Therefore, they have the inherent nature of being political biased and ideologically influential. The fantasy that "technologies should be un...

Week2: ImageNet and MobileNet  Jianhao Ma ImageNet My initial impression with ImageNet is about how slow it is. But I thought that makes sense regarding it is a non-profitable research project with a huge dataset. I tried to find how big the dataset is and figured out that the statistics hasn't been updated for 10 years. Later I realized that not only the search engine lacks consistency, the tree view is also taking forever.  As I browsed through ImageNet, I started to have a sense that it's one of those early Image Classification project that's been forbidden by time. Can't neglect that there are many privacy concerns, especially under "person" synsets. I think that they would have be more careful when selecting synsets of illegal occupations and minority groups if it's a recent project. But I wouldn't blame that on ImageNet because that's the common theme of fabricating any datasets with all resources on the Internet. It's too much works for ...

Week 4: Play-test Feedback Tested Puzzle - File#13 Jigsaw Puzzle Common Questions: Will this puzzle use the name of player for better immersion? If so, how to achieve that technically?  Average Times taken to solve the puzzle: 4 mins. Game Master Script: Now that you have completed several tasks given by the professor. But he didn't tell you where to archive these files. When you pullet out the cabinet to try putting them in place, and found this:  You found the sticker underneath seems like your name. Driven by curiosity, you peeled off the sticker above it, and it indeed is your name. But the folder was empty. You recalled that the trashcan in professor's office usually has shredded paper in it. You were so eager to know why is your name appeared in the cabinet of experiment journals, that you decided to take a chance and discover the trashcan. As expected, there were pieces of papers in the trashcan... After finishing the puzzle - You were...

For this week's assignment, my task was to design a prototype puzzle that involves compartmentalization elements. This puzzle requires players (at least two) to compartmentalize their surroundings to solve most efficiently, though one player can solve the puzzle anyway. Setting: Two players were locked in the lab triggered by unauthorized intrusion of a protected container (safe? drawer?). This puzzle is one of the alternative ways to get out. On the wall outside of the professor’s office, there is an emergency evacuation map. On the map, the players can find out where are the two doors that locked them in this floor. But as stated above, these locks are triggered by an electrical alarm system around the safe. And this trigger-lock system is modded by the professor himself. On the map, there was this electrical room where all the maintenance happens. (Just any room, I haven't decided yet at this stage) On the professor’s desk, There is a calendar where the profess...

Synth Space Odyssey https://editor.p5js.org/roger1mjh/sketches/ME3C2-2f Inspiration So the inspiration of this game is from a classical arcade game Raiden. Our idea was to make a replica of this game in P5 with melody elements. Live Demo https://www.youtube.com/watch?v=7iDwR3JnPcI&feature=youtu.be Description Originally, we wanted something more absurd - a noisy baby killer game with same mechanics. A baby crawling on the street use loud cries to eliminate every obstacles on the way. We even designed sound and animation for it and plan to use sampler to generate crying sound at different pitch. Sadly, the generated pitch are terrible, and we eventually returned back to the safe choice - midi synth. Since sounds of midi synth do not make sense to the crying baby theme, we returned to spaceship and asteroids. Overflow First of all, we had the trigger attack and release function tied to mousePressed function. Then, we map the frequency with mouseX. After that, ...

1. Skiing Game I was inspired by those classic 2D skiing and mountain motorcycle games. The player control the balance of the character to ski through the "mountains"(melody wave). 2. Pottery Wave Maker The user make potteries and place them in sequence. The camera or lidar scan the shape of the potteries and make a music based on them. 3. Flies An interactive melody game. Two flies fly side by side simultaneously, each represent a melody that are similar to each other. After listening to the original melody once, the user will be asked to choose which one of the two flies represent the right melody.

A11: Nobody's Fault but Mine Drawing of Melody The Song I picked is Nobody's Fault But Mine by Led Zeppelin. Four different colors represent the four main riffs of this song. The spray paint has nothing to do with melody, I just thought it fits the distortion and drums quite well. Link to the song: https://youtu.be/gqQelCIGYMI Riff 1: 0:00-0:24 Riff 2: 1:08-1:40 RIff 3: 3:00-3:30 Riff 4: 4:06-4:20

A10. Clap Beats Jianhao Ma, Shihao (Tiger) Zhang Links: https://editor.p5js.org/roger1mjh/sketches/rJMbn4UG Video Demo: Inspiration: Originally, we were thinking about using poseNet and camera to make a gesture interactive random beats generator with Markov Chains. However, we thought that almost every IMA student knows how to do gesture control in P5, and we have done similar projects before. Therefore, we switched to sound control. The idea is: the tempo of the beat is determined by the frequency users clap their hands.  Design Decisions: The music style we aimed for is Lo-Fi. At first, I imported 26 tracks of samples and categorized them into five arrays. We tried to incorporate some visual designs like changing colors and shapes based on the last track played and the array it belongs to. Sadly, Shihao was sick when we moved on to that part so we just left it as future works. How does it work? So the core part of the project is to measure the tempo of user...

My Box Sequencer and Circle Visualization A Dumb GIF illustration: I just followed the steps we learned in class and stylized the box sequencer a little bit by shrinking the size, add strokes, and changing colors. Those were the easy part. Then I recalled that Luisa encouraged me to try making a circular sequencer with arc(x,y,w,h,start,stop). I felt confident about it because I've used arc several times in P5. However, when I actually coded it, I found it's more complicated than I thought.  The first difficulty assigning orders. At first the bigger arc overlapped the smaller ones and hided them underneath. I fixed that through changing the w and h from adding up to decreasing down.  The second challenge was the mousePressed function. It's doing not simple math with mouseX, mouseY, and w anymore. In order to make it work, I have to create a class of arcs and assign the arcs that represent each drum note to objects of that class. It's too much labor to do ...

Assignment 7 Music: Dance Yrself Clean - LCD Sound System I remake the drums based on sheet music to begin with. Then I start searching for Electric piano sounds, however, there's no specific sound samples for every key. Therefore I found this bridge on HookTheory: https://www.hooktheory.com/theorytab/view/lcd-soundsystem/dance-yrself-clean So I recorded this bridge and put it in my code to run them in crossrhythm. Here is my final code: https://editor.p5js.org/roger1mjh/sketches/UnIiJikx

Design Ideas for Rhythm Browser Sequencer A web-based sequencer. Each browser window represents a note. By rearranging and resizing these windows, the user can play with the beat they generated. Typos Syncopation Machine Also a web-based interactive player. Inspired by KeyDrummer. It will presents the user a sentence. The player need to blind-type the sentence in the text box. Then it will detect how many typos are there and add syncopation to the music.  Shake Phone Syncopation An interactive app for mobile devices. The player add syncopation to the music by shake their phones. The app will detect the accelerations and directions and add syncopation based on them.

Chapter 2: Rhythm, Loudness, and Harmony Reading Notes: Rhythm: lengths of notes tempo: pace of a piece of music meter: the way in which the pulses or beats are grouped together "perfect fifth": notes that are either seven semitones higher or lower in pitch are considered a perfect fifth notes from the starting note a whole note: standard duration, lasts 4 beats beat: the basic unit of measurement of in a musical piece, tactus Pitch and loudness do not actually exist in the world, they only exist in the mind Key: the tonal context for a piece of music consonance: pleasing sound chords and intervals dissonance: unpleasing ones melodic transposition: playing a song of music in a different key or with different pitches The Gestault School: a group of psychologists who studied people's cognition towards grouped elements as a whole "Grouping" Response: This chapter introduced music terminologies such as rhythm, tempo, meter, etc... and demonst...

Roundabout by Yes Briefly introduction: This is one of the most iconic song by the prog rock band Yes. The first 50 seconds of the song became a meme music because of it's reference in JoJo's Bizarre Adventure .  https://youtu.be/lvANpEr3XbA Reasons I like it: The bass riff is amazing and catchy The complicated and delicate composition of different instruments give this piece great layering The rhythm is simple and memorable, but presented in a intricate and skillful way I watched so many meme videos of this...

Catalog Of Attributes Andromeda by Gorillaz 130 bpm Sound elements: Mostly Synthesizer(bass, electrical keyboard), male vocals, female vocals Harmony: A Minor Melody: Beats of short and responsive bass, Middle Range covered by male vocal in most of the time, Tremble are mostly electrical keyboard Most of the song is in the same rhythm pattern(130 bpm) directed by bass, the last 30 seconds is half as quick as the rest of the song Overall a dynamic and immersive song with strong rhythm and crowded contents, a decent dance music Concept For an interactive visual design, I was inspired by the Gorillaz's official video of this song on Youtube. It does a fine job creating a steady visual representation that fits the theme and vibe of the piece. My idea is to add a control panel on top of that. There would be three toggle bars, each determines "bass", "vocal", and "electronic", corresponded to bass, midrange, and pitch. By toggling ...