Making Art in the Age of Algorithms

Jan 18 2019 | By Allison Elliott

As any engineer can tell you, their field is a natural fit for makers, designers, and all manner of creative thinkers. That’s particularly evident at Columbia Engineering, where creativity permeates all levels of the curriculum—as well as research and extracurricular arenas. Here artists have embedded in labs for unique initiatives, such as developing XR storytelling tools, or held residencies that bring together engineering with other disciplines to trigger new ways of understanding

Recently, an interdisciplinary group of students added a new reading group, “Art in the Age of Algorithms,” to the list of initiatives connecting art, technology, and engineering. The group, which debuted last semester, is spearheaded by Columbia Engineering student Katy Ilonka Gero, a PhD candidate from computer science, and Arts & Sciences student Eamonn Bell, who is earning his doctorate in music theory.

Designed to explore how the arts intersect with computation, code, and theory, the group focuses on the cultural and ethical ramifications of new processes and practices that computation is making possible. A reading group is the ideal venue to tease out these interlocking issues, says Gero, because unique insights stem from interdisciplinary dialogue.

“You might have the same thoughts as others in the lab, but it’s important to see new viewpoints, to read a technical paper or critical theory,” she said.

For Gero, who spent four years in the startup world before entering graduate school, the group was also an opportunity to combine her love of literature and computer science. At Columbia, she studies with Assistant Professor Lydia Chilton, an expert on computer-aided design and creativity, while off campus she works with the New York-based School of Poetic Computation, an artist-run organization that explores computational approaches to art and design.


A simulation shown by Computer Science PhD student David Watkins at the forum helps robotic hands learn to grasp real objects.

For his part, Bell says in Gero he found a collaborator with a deep appreciation for the inherent crosspollination between the disciplines.

“I needed someone with an area of expertise who also cared deeply about other domains,” said Bell, who initially conceived of the group.

Together, they developed a structured, yet informal, reading plan and meeting schedule for the semester, which ultimately brought together ten graduate students from engineering, science, and the humanities. Each session was organized around a series of themes, from defining algorithms to conversational agents. Readings spanned diverse areas such as bias in training data; algorithmic music in the work of trombonist, composer, and Columbia professor George Lewis; chat bots for racial justice education; and even crowdsourced dating experiments.

The expansive approach resonated with Liz Bailey, a graduate student in chemistry with an undergraduate degree in studio art. She came away with a new appreciation for the complex issues surrounding the topic. “What came up is that algorithms aren’t always neutral,” she said. “They are reflective of what we put in.”

Bailey was among a number of group members who presented lightening talks in December at the group’s first public forum, all of which were inspired by issues raised over the course of the semester. In her short lecture, Bailey interpreted several works of art currently on display at the Whitney as part of a timely exhibit, “Programmed: Rules, Codes, and Choreographies in Art.” Describing how a Random Walk algorithm helped generate a sculptural piece called Broken Volume (10 L) by Cheyney Thompson, she shared how the sculpture, which inadvertently broke during installation, typifies some of the unintended consequences when algorithms confront the limitations of physical reality.

She wasn’t the only one to engage directly with computer-generated artifacts. Nick Montfort, a professor in MIT’s Comparative Media Studies/Writing program, opened the forum with a keynote address in which he shared examples of his own algorithm-generated poetry. Jeanne Devautour from the French and romance philology department collaborated with Samuel Boury, an independent researcher, to apply a Random Walk algorithm to the practice of Oulipo poetry, a movement that employs the use of specific, structural constraints in the creation of poems.

Other group members explored themes ranging from how media and entertainment companies repurpose customer data to how data on climate change can reframe the way we see the world.

David Watkins, a PhD researcher in Peter Allen’s Columbia Robotics Lab, shared how improved simulation methods were leading to great leaps in researchers’ ability to advance robotic manipulation, while Oscar Chang, a PhD student in Hod Lipson’s Creative Machines lab, expanded the audience’s perception of creativity with an inquiry into how automated drug discovery could help solve the pressing problem of antibiotic-resistant strains of bacteria, otherwise known as superbugs.

Audrey Amsellem, a teaching fellow in the music department, addressed music-streaming platform Spotify’s recent partnership with genealogy site Ancestry DNA to offer customers a music playlist based on their DNA. She invited the audience to think about the ramifications of conflating cultural preferences with genetics and how companies use our data to create taste profiles of consumers in order to “sell us back to ourselves.”

Elizabeth Case from earth and environmental sciences taught a mini-workshop in creating “glitch art,” or digital art that is purposefully manipulated through data corruption to create new and compelling visuals. Case started corrupting her own naturescape photos from Alaska in order to produce images that are not just aesthetically interesting, but also bring attention to how these landscapes are threatened by climate change and environmental degradation. Going forward, she’ll embed an additional layer of meaning by using climate data to create the glitches.

One of the benefits of their critical framework, Bell said, is how it operates as “a scaffold for thinking outside the box and bringing people together who wouldn’t usually get together.”

Bell’s own talk penetrated deeply into the definition of an algorithm – from pragmatic to computational angles. In her presentation, Gero explained how visual editing tools like Photoshop and Instagram filters changed the way we worked with images. She challenged participants to consider what the Photoshop for writing could be.

Next semester, the co-leaders plan to build on the success of the project, expanding their programming and reach. In doing so, they hope their efforts will inspire other students to start similar collaborations around topics of interest.

“It takes time, not expertise,” said Gero. “If anyone wanted to do something like this, lots of people at Columbia are happy to support you.”

In the future, Columbia Engineering is developing additional interconnected programming with Arts & Sciences, including collaborations with art students to create work that communicates the urgency around sustainability and investigates clean energy storage.

The public forum was sponsored by the Center for Science and Society—which also funded the reading group itself through a seed grant—along with the Brown Institute for Media Innovation, and the Columbia University Data Science Institute.

Eamon Bell, co-founder of the reading group, gives a lightening talk at the December forum.