This camera turns photos into AI poetry

Have you ever stood in front of a redwood tree and wondered, “Wouldn’t it be great if it was poetry instead of a tree?” Neither Joyce Kilmer. However, Kelin Carolyn Zhang and Ryan Mather decided to bridge the gap between AI technology and poetry with their fascinating idea – Poetic camera. The open source device combines cutting-edge technology with artistic vision, leading to a piece that pushes the boundaries of each fields.

At first glance, the Poetry Camera looks like just one other gadget within the ever-changing landscape of digital devices. However, upon closer inspection, it turns out that this isn’t any bizarre camera. Instead of simply capturing images, the Poetry Camera takes the concept of photography to a brand new level by generating thought-provoking poetry (or, well, as thought-provoking as AI poetry can get) based on the visuals it encounters.

Zhang and Mather, the creators of this project, seamlessly combined their knowledge of technology and art to create a tool that’s technically impressive and aesthetically fascinating.

The Poetry Camera story began as a private passion project for Zhang and Mather, who were captivated by the thought of ​​combining their love of technology with their appreciation of art. In the early stages of development, they spent countless hours tinkering with various components and experimenting with different AI models to bring their vision to life.

“The beginnings of the project date back to the moment I gained access to GPT-3. My first instinct was to play Dungeons & Dragons on it because I’m a nerd. I thought “if this thing could play Dungeons & Dragons, that would be impressive.” And yes, it worked. This was back when engineering needed to be done quickly. So it took some grease to make it work. But I also had the idea to make a camera as part of the project,” says Mather. “What if you picked up a camera, but it was a reaction to Instagram culture? What happens if text appears instead of a photo? … Everyone prefers the book version to the movie, so that’s what it’s like when it comes to capturing moments.”

As they perfected the prototype, Zhang and Mather began sharing their creation with family and friends at social gatherings. The reactions they received were simply amazing. People were fascinated by the concept of a camera that might generate poetry from what it saw. The device quickly became the focus, sparking full of life discussions and igniting the imagination of everyone who encountered it.

These initial reactions provided strong motivation for Zhang and Mather to further refine their invention. They realized that the Poetry Camera had the potential to bridge the gap between technology and art in a way that appealed to people. As word spread concerning the Poetry Camera, the duo was inundated with inquiries from curious individuals who desired to experience the device for themselves. This interest led them to contemplate turning their project into a possible industrial product, bringing the magic of the Poetry Camera to a wider audience.

Technique

At the guts of this progressive device is the Raspberry Pi, a single-board computer the dimensions of a bank card that packs loads of power. This small but powerful component serves because the brain of the poetry camera, allowing it to capture images and communicate with OpenAI’s GPT-4 to generate poetry.

The Raspberry Pi captures the image after which uses computer vision algorithms to research the visual data. AI models then interpret the image, identifying key elements, colours, patterns and emotions within the frame. This information serves as the idea for the poetry generation process.

Zhang turned the camera on our Zoom interview:

And out got here the poem:

Poem generated and printed by Poetry Camera (transcript below). Image credits: Poetic camera

Behind the lens, a person corrects his eyesight
Within the square boundaries of pixel space.
Magnification of faces in a virtual place,
He left the rat race an extended time ago.

Daylight penetrates through faded curtains,
He holds lukewarm coffee in his hand.
Conversations drift across the virtual border,
His mirror image appears on the screen.

There are thin books on the shelf, quiet chatter,
A flowering plant, the one living matter.
With the glasses on, the home becomes his calm beater,
Within these partitions his world dissipated.

On this April morning in 2024.
New normal, worn calmly.

Zhang theorizes that the reference to “leaving the rat race” is a reference to my Hawaiian shirt.

The creators tell me that Poetry Camera’s output is just not limited to 1 format. Because the device is open source, users can select from quite a lot of poetry forms – akin to haiku, sonnet, or free verse – depending on their preferences and talent and willingness to interact with the source code.

As the Poetry Apparatus evolves, Zhang and Mather are exploring latest ways to expand its capabilities. They are continually refining and curating AI models, expanding device suggestions, and experimenting with advanced natural language processing techniques. The goal is to create a tool that generates poetry and engages in meaningful dialogue with users, fostering a deeper connection between technology and art.

The intersection of art and technology

The Poetry Camera is an affidavit to the incredible potential on the intersection of art and technology. Using the ability of artificial intelligence and machine learning, Zhang and Mather have created a tool that not only captures the great thing about the world around us, but additionally interprets it in a way that might be surprisingly moving. The technology is straightforward, but I really like the accessibility of Poetry Camera. It’s easy to dare to dream and push the boundaries of what technology can achieve.

Screens dominate our day by day lives; The Poetry Camera offers a refreshing departure from the norm. Instead of counting on a digital display to showcase your poetic achievements, the device uses a more tactile and immersive approach to interaction. The generated poems are printed on paper, making a physical embodiment of the artistic experience. With Poetry Camera, the temporary nature of art is pushed quite far.

Another sample poem. Image credits: Poetic camera

“We do not save any images or poems in digital form. There are several reasons: First, it is easier. Two: privacy. Third, it gives the poems additional meaning if they are something like ephemeral artifacts. If you lose it, it’s gone,” explains Mather. “Now everyone has a camera in their pocket connected to a mobile phone. We wanted to do something completely different.”

We’ve seen latest user interfaces with products like Plaud and Humane Ai Pin – and this device continues in the identical vein. It allows users to disconnect from the constant flow of visual stimuli and interact with Poetry Camera in a more mindful and contemplative way. I don’t desire everyone to be an art student, however the visceral experience of holding a printed poem in your hands is an invite to reflect on the words, appreciate the great thing about language, and develop a deeper reference to the murals.

The design philosophy extends beyond the device, impacting the whole user experience. Zhang and Mather fastidiously crafted the Poetry Camera to evoke a way of wonder and discovery. Capturing a picture, waiting for the AI ​​to generate a line, after which receiving the print creates a way of anticipation and surprise, increasing overall engagement with the device.

Reflecting the longer term of artificial intelligence and private computing

Zhang and Mather’s Poetry Camera design philosophy reflects their broader vision for the longer term of non-public computing and artificial intelligence. They imagine that technology must be seamlessly integrated into our day by day lives, enhancing our experiences without overwhelming us with constant stimuli. By making a device that prioritizes simplicity and artistic expression, they challenge the established order and pave the best way for a brand new era of technology.

“I feel poetry cameras are a microcosm of what is going to occur to many industries with AI. Poetry cameras are different from a standard discipline: You’ve never met someone who said, “Oh, I’m a poetry photographer for corporate events,” laughs Zhang. Poetry Cameras are somewhere between this recognized field of photography and poetry. This is a brand new, strange thing. Most importantly, watching people interact with them is fun: people find childlike joy of their playful personality.

What’s next?

As Poetry Camera continues to captivate audiences and generate buzz, Zhang and Mather are fastidiously considering the longer term of their progressive work. While the device started off as a private passion project, the overwhelmingly positive response led them to explore the potential of making it commercially available to a wider audience.

“After the first 100 questions people said, ‘We don’t sell this,’ but after asking 101 questions we started thinking about it in more detail and wondering if we should make it available to people,” Zhang reflects. “But at the same time, it’s an art project, you know? Our initial reaction was to leave capitalism out of it.”

However, consistent with their commitment to sustainability and conscious consumption, Zhang and Mather take their time before committing to mass production. Instead, they’re taking a more measured approach, with limited drops possible on products that favor quality over quantity. By fastidiously controlling the production process, they strive to be sure that each poetry apparatus is made with the utmost attention to detail and meets high standards of performance and sturdiness.

I challenged the team on the necessity for a hardware product: phones have cameras, web connections, and screens. Wouldn’t it’s more sustainable to make use of equipment that already exists?

Another example poem, within the background the one that inspired it. Image credits: Poetic camera

“The most sustainable solution would be to do nothing at all,” agrees Zhang. “But we initially started it as open source DIY projectand we had all the instructions online; it was just hard for people to get all the parts they needed. Initially, we toyed with the idea of ​​creating a cardboard casing. However, I’ve noticed that part of what makes people love the product and want to play with it is the product’s aesthetic, refined in a Poké Ball style. I think that for the Poetry Camera to come to life in its full form, it should be a product that is refined in appearance. I don’t know how to approach it in terms of sustainability, but it’s important to us.”

Looking ahead, Zhang and Mather imagine a future during which the Poetry Camera is just not only a novel device, but additionally a catalyst for change in the best way we interact with technology and art. They hope that their work will encourage others to explore the intersection of those two fields, pushing the boundaries of what is feasible and creating latest types of expression.

I really like living in a world where enthusiastic hobbies intersect with art and technology, and this project suits perfectly into the Venn diagram.

“For me it continues to be art since it lets you express. It is a tool for shaping the world we would like to see, a world where people can play with latest technology: artificial intelligence is just not all doom and gloom. And it is not just that artificial intelligence imitates humans in a deeper, faster and worse way,” Zhang concludes. “It’s about creating something completely new with new technology and bringing out that inner child again. This project is not about making money – it is about making this childhood miracle possible again.”

If you wish a more in-depth take a look at the camera in motion, TikTok appears to be the most effective source.