# Picture This: Open Source AI for Image Description

DevFeed: [Picture This: Open Source AI for Image Description](<https://devfeed.tech/articles/picture-this-open-source-ai-for-image-description-1707.md>)

Original publisher: [Read original article](<https://fly.io/blog/llm-image-description/>)

Published: 2024-05-09T00:00:00Z

Content type: article

Language: en

Sources: [The Fly Blog](<https://devfeed.tech/sources/the-fly-blog.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [alt text](<https://devfeed.tech/topics/alt-text.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [alt-text](<https://devfeed.tech/tags/alt-text.md>), [cdn](<https://devfeed.tech/tags/cdn.md>), [close-to-users](<https://devfeed.tech/tags/close-to-users.md>), [deploy-app-servers](<https://devfeed.tech/tags/deploy-app-servers.md>), [docker](<https://devfeed.tech/tags/docker.md>), [elixir](<https://devfeed.tech/tags/elixir.md>), [fly](<https://devfeed.tech/tags/fly.md>), [fly-io](<https://devfeed.tech/tags/fly-io.md>), [heroku-alternative](<https://devfeed.tech/tags/heroku-alternative.md>), [heroku-competitor](<https://devfeed.tech/tags/heroku-competitor.md>), [hosting](<https://devfeed.tech/tags/hosting.md>), [i](<https://devfeed.tech/tags/i.md>), [image](<https://devfeed.tech/tags/image.md>), [llms](<https://devfeed.tech/tags/llms.md>), [networking](<https://devfeed.tech/tags/networking.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [postgresql-clusters](<https://devfeed.tech/tags/postgresql-clusters.md>), [servers](<https://devfeed.tech/tags/servers.md>)

## AI overview

This article presents AI-generated image descriptions as an important accessibility aid for blind internet users. It describes how large language models can identify objects, describe scenes and menus, and summarize technical diagrams, while framing an extensible open-source project as something developers can build over a weekend.

## Source excerpt

I'm Nolan, and I work on Fly Machines here at Fly.io. We're building a new public cloud--one where you can spin up CPU and GPU workloads, around the world, in a jiffy. Try us out; you can be up and running in minutes. This is a post about LLMs being really helpful, and an extensible project you can build with open source on a weekend. Picture this, if you will. You're blind. You're in an unfamiliar hotel room on a trip to Chicago. If you live in Chicago IRL, imagine the hotel in Winnipeg, the Chicago of the North. You've absent-mindedly set your coffee down, and can't remember where. You're looking for the thermostat so you don't wake up frozen. Or, just maybe, you're playing a fun-filled round of "find the damn light switch so your sighted partner can get some sleep already!" If, like me, you've been blind for a while, you have plenty of practice finding things without the luxury of a quick glance around. It may be more tedious than you'd like, but you'll get it done. But the speed of innovation in machine learning and large language models has been dizzying, and in 2024 you can snap a photo with your phone and have an app like Be My AI or Seeing AI tell you where in that picture it found your missing coffee mug, or where it thinks the light switch is. Creative switch locations seem to be a point of pride for hotels, so the light game may be good for a few rounds of quality entertainment, regardless of how good your AI is. This is big. It's hard for me to state just how exciting and empowering AI image descriptions have been for me without sounding like a shill. In the past year, I've: Found shit in strange hotel rooms. Gotten descriptions of scenes and menus in otherwise inaccessible video games. Requested summaries of technical diagrams and other materials where details weren't made available textually. I've been consistently blown away at how impressive and helpful AI-created image descriptions have been. Also... Which thousand words is this picture worth? As a bli