Christina Chen

GradEx 108

04/27/2023 At the end of the year, OCAD has this exhibition of graduating students' work, called GradEx. My final project, Crystal type, is about making fonts by growing crystals and inputting the letters into a machine learning model. For GradEx, I wanted to set up a space where people could use the font, so I was thinking about how I wanted people to interact with it.

A keyboard is kind of boring, and there’s too much that could go wrong with keyboard shortcuts. So I wanted to have a mic instead, that you could talk into, and your words would show up on the screen. Whisper API had just come out, so I wanted to try it for speech-to-text. I was looking at examples of how to implement it, but most of them were like, you record something and then send that recording to the API. I needed something more real time. Thankfully, I found this React hook.

But when I tried it, I was surprised at how high my API usage was. This is because it sends the entire audio stream from the beginning based on "timeslice" seconds. So even though Whisper API is 0.006 cents per minute, I actually paid 1 cent for a few seconds of testing. Because, for let’s say a 30 second recording, I wouldn’t be paying for 30 seconds. I'd be paying for 1 plus 2 plus 3, all the way to 30. And after some napkin math, this over the five days of GradEx would cost me over $4000.

I decided to instead just leave the website running in a loop where it would record for 5 seconds, send the recording to Whisper, display whatever got transcribed, and then repeat.

04/29/2023 I was going to put up three posters above the website, but in my head, I imagined that there was going to be a white wall. When I got to the space I was designated, I realized it was glass, and I wasn't really vibing with that. I measured the space and I found that covering it with 40 tabloid size posters would work. But that meant I had to figure out what to put, four days before the exhibition. I did have some photos, screenshots, and of course my fonts. It was actually a lot of fun, doing the layout.

I also wanted to go through each poster briefly. These four are showing the series of fonts, and the progression from Grown to Traced to Learned.

These are just the Traced letters, Traced with alternate characters, variable weights, the In Between font, and what I got from DeepVecFont.

These are photos of when I was growing the crystals submerged in water, how they turned out with wire, pipe cleaner, and when I put them into a bowl.

This row is mostly DeepVecFont outputs, so all the different X’s overlayed on top of each other, the letters with crystal pieces, and more just playing with the letters.

Down here is some process, so the DeepVecFont paper, and a screenshot from when I was making the alternating font. I wasn't sure about how I felt about this font initially, but I kept going back to look at it. Yihui was saying it's like the feeling you get when your work turns out unexpected, and I was being a human Whisper API, transcribing it into my font proofing.

There's also a really big screenshot of my Are.na channel, which has notes and things I collected while I was working on this project. And also where I got a lot of quotes for the next posters.

The envelope quote is from Kurt Vonnegut. Some of the text is in the Traced font, and for some of the sentences, I used only the crystal formations, which is this alphabet next to it. It was inspired by David Carson's use of Dingbats to set a full interview in. I like this quote because you could go online and buy a hundred envelopes, and save time and money, which is a lot of the time, is what technology is about. But we're not alive to just save time and money. And sometimes you just wanna dance around and go outside and get one envelope. In a shorter way, you could also say, "What's the point if we can't have fun?" which is also the title of a piece by David Graeber.

Another side of this project was also thinking about the way we communicate. McLuhan's Medium is the Massage was really influential, and I was thinking about fonts as a medium. This is in Traced, and the images being masked are actually photos of growing the crystals. There is one of the pages from Facebook's Red Book that they gave out to employees back in 2012, overlayed with some of my notes. And the famous quote, "Technology is the answer, but what was the question?" with the In Between font on top, and Traced font on the bottom. There is also a QR code, and a layout of all the posters in order, which I put in the bottom right of the wall, so I could reference it when I was putting up the posters.

05/01/2023 After I finished that, I got hit with another curveball. The smallest screen that was available was a big TV, and the time window had passed for me to get a desktop monitor. I didn’t want to cover up that much of my posters, but luckily no one was assigned to the space next to mine, so I could put the TV there.

05/02/2023 But, the TV with the glass felt disjointed from the wall with the posters. So I decided to print these out and put them up behind the TV.

Then I started setting up my Whisper app. Originally, I was using a Chrome stick. You plug it into the TV, and it turns it into an instance where you can browse the web. I opened my website, connected my mic, and tried it. But so many things went wrong. At first, the font wasn’t even loading. But I fixed that by putting the file in a different folder. Then, I opened the console and I found that it wasn’t even sending the transcription request to Whisper. It might be that Chrome stick doesn’t work very well with interactive elements. For example, even Netflix's interactive Black Mirror: Bandersnatch doesn't work with Chrome sticks. I sat there doing some troubleshooting for a few hours.

But the exhibition was the next day, so I decided to pivot and connect a laptop to the TV instead. Which meant I had to make a box to hide the laptop. Fortunately, there was scrap poster board on the Advertising floor I could use.

05/03/2023 It worked out pretty well, because the table was kind of low anyways, and I needed the mic to be higher up. After covering it with some black paper, it was ready to go.

At the opening night reception, we got free food! I talked to 0429, her friends, and 0430.

My parents, 0420, 0431 from Western, and 0432 from Hack Western came! 0399 introduced me to Leah from Leah's Fieldnotes. A lot of people told me, "I can see why you got the medal." And that they liked my outfit. Some people were taking pictures of my work like an Instagram background, others were reading my booklets. One person shook my hand and asked to take a picture of what I wrote, saying it was really relevant. When we were done for the day at 11 pm, I went to go eat congee with 0399.

05/04/2023

During the five days, our class took turns watching the Digital Futures space so that there was always at least one student there. I signed up for the morning slot, when it was quieter. I sat at the back on my computer, sometimes writing down things I heard people say as they interacted with my project.

Some high school groups came by, and a dean asked me "Why would you study Digital Futures compared to Computer Science?" My Creative Economies professor also brought a group to our area, and I explained my project to them. "This project reminds me of the other Graphic Design Medal Winner project, you should talk to him." I got that a few times. Donald and Richard came down to my exhibit and started talking about printers.

I visited all the floors twice! And bought prints with 0399. Some of my favourite works I saw today were by Youngmi, Agnes, Donald, Colin, and Alison.

It was Digital Futures night. There was free food and awards! I got a bouquet of flowers. Shoutout to Kahani's game and Ziqi and Weiqi's co-creation studio. A lady asked me about how I grew crystals with Borax, and I talked to a mayoral candidate. Three people said they liked my pants, and did I make them myself? (No, I didn't.) 0433, 0434, 0435, and 0436 also visited. After, I went to the post office at 8:30 pm, then ate at Bao with 0399.

05/05/2023 This morning, there was someone who stayed at my work for a long time, playing with the computer, reading, and looking at posters. Two separate people read "Technology is the answer, but what was the question?" out loud. There was a girl walking around and explaining projects to a group, but she was getting everything wrong. She looked at Aimee and Carole's pet owner community app, and said "This is a UX project, but the users are pets," then said "This is an urban planning project," about Prayag's work. She looked at mine, "Oh... I think this is AI."

"Let's rock and roll people... it did come up there. That's pretty good. You have to talk for five seconds."

"It looks like they grew letters, see the bag of letters? See, it's crystals. I think they grew the letters with crystals, and then traced, and then I think the computer's learning to... I don't know, I have to read about it."

"Mom, that's an X?"

"Damn what... that's kinda cool."

"'Well,' that was actually accurate. That's what I said."

"This is really well written."

"You made that? That's yours?"

"This is so entertaining. I could stand here for hours." Looks behind monitor. "How is that working?"

"Est-ce que tu parles français?"

When they said that, "Do you speak French?" showed up on the screen. Lots of people try different languages, which sometimes works out really well because Whisper does translations.

I had a few conversations about my work today. Someone was asking about Figma, and we were talking about FontForge. They asked, if I didn't use crystals, what else would I have wanted to use? What part of the project am I most excited about? And some questions about model training and datasets. I talked to someone about beamforming microphones and Raspberry Pis. Then someone was telling me we should stop using QR codes for our projects, because they can be vehicles of malware. There was also someone who visited the Digital Futures space twice, they asked "Is this OCAD?" and "Are you graduating?"

Today I was also running around making more booklets. For lunch, I tried the new pho place at the Grange with 0399! I saw 0437 and 0438, and 0306! 0025, 0013, 0013's friend, 0022, and 0439 also came. We got Fruiteao, and I brought them to all the floors, 0430, Animation, Materials, and new places I hadn't gone. I really liked Sophie and Liv's work. 0429, 0440, and two others did Leanna's life coaching karaoke, and sang "Castle on the Hill." At 8:30 pm, I had dim sum with 0399, 0022, and 0439, and then Sugar Marmalade with 0399.

05/06/2023 "Hello, how are you doing today?"

"Speech-to-text... OK, not the most innovative thing."

Picks up booklet. "That's really cool." Continues reading. "...OK very cool."

"Is this programming?"

"Hello hello hello... Is this on the whole time?" Looks behind the TV. "Is there an Alexa underneath or something?"

"That is so cool, it's like AI."

"Oh look, it's cool. It said hello... that's cool, I like that... oh it said 'That's cool, I like that.' OMG it's still listening!"

Today I saw 0429 and talked to 0440 about UX. I played Michelle's and AJ's games, finally. I met 0399's friend and mom. We sat in the theatre and watched animations. I also found new rooms – ceramics, jewellery, Integrated Media, and Materials. 0026 came with 0441 and 0442, and brought me Hattendo! I showed them the Environmental Design floor and other parts of OCAD.

Then I went to CUTC to share my work. I talked to a few people, the crowd was mostly all CS. People asked me technical questions about the model, someone told me about Deep Floyd, another person working on a mental health initiative reached out to me for potential collaboration opportunities, and others asked me about my business model. (I don't have one.) I also caught up with 0443 from Western!

05/07/2023

"'What they think you said'..." Makes random indistinguishable sounds. Laughs at the output.

"Hello tables!"

"This is cool, it's like an AI generated font."

"Oh it's you right? Yeah awesome... oh it listens, does it?"

"こんにちは。"

"I'm fighting you. I'm human."

"How dare you."

Today someone was in Ziqi and Weiqi's room, telling them about an opportunity, but they couldn't tell if he was 画饼. I stopped by, trying to hold in my laughter, and he said, "Does your friend have... value?" Then in the theatre, Kim's animation was playing, and the person behind us said something like, "She'll go really far," not knowing that Kim was right there. For lunch, I had Fruiteao and Korean food from the Grange with 0399. 0444 came with her church friends, and 0445 and 0446 came! 0424 also brought his friend.

A lot of people helped me de-install. Then me, 0423, and 0399 went to eat JBBQ!