Securing And Exploring Photos With AI: What's In The Background Of PhotoPrism?

Welcome to this week’s Deep-fried Dive with Fry Guy! In these long-form articles, Fry Guy conducts an in-depth analysis of a cutting-edge artificial intelligence (AI) development or developer. Today, Fry Guy is exploring PhotoPrism, an AI-powered open source photo management application. We hope you enjoy!

*Notice: We do not gain any monetary compensation from the people and projects we feature in the Sunday Deep-fried Dives with Fry Guy. We explore these projects and developers solely for the purpose of revealing to you interesting and cutting-edge AI projects, developers, and uses.*

Have you ever sifted through thousands of photos on your phone, trying to find that one memorable shot of you and your friend? Wouldn’t it be so much easier if you could just search for a photo by person, place, or description?

PhotoPrism, one of the hottest projects in the AI space, is a powerful photo management tool that prioritizes privacy and user control.

WHO’S IN THE BACKGROUND?

Michael Mayer and Theresa Gresch are the masterminds behind this photo storage solution. Mayer grew up in southern Germany with an early interest in computer science. He has a background in IT and has experience with business applications and web services. Gresch earned a master’s degree in neurobiology and began to take an interest in AI due to the industry overlaps. Early in her career, she worked as a neuroengineer and eventually found her way into software product management.

PhotoPrism was born through Mayer and Gresch’s personal experiences with managing their large, unsorted photo libraries. They explain, “We had a lot of pictures on our computer that were unsorted, and we thought it would be faster to write the software than to sort it manually.” In recognizing the need for a privacy-centric and user-friendly photo management solution for themselves, they began to develop PhotoPrism.

AN AI PHOTO STORAGE SOLUTION

With PhotoPrism, users can easily organize and manage large amounts of photos without compromising their privacy or data control.

PhotoPrism has different pages for the user to manage their photos. This includes a section where the user can search through all their files, a section sorted by objects, a section sorted by people, and a section sorted by locations that are clearly displayed on an interactive map. Photos can also be sorted into a category by nonphotographic quality content, such as memes and screenshots.

The program uses AI to automatically detect and analyze photos, classify images, and perform metadata extraction. By doing this, the system is able to sort your images based on customizable data like objects, people, locations, and more. This also allows users to search and browse their photo library based on different criteria such as keywords, time, colors, quality, and descriptions, saving users time and effort by automating the process of photo organization.

Recently, PhotoPrism released a facial recognition feature, which uses AI facial recognition to identify people and organize the photo library accordingly. In this way, users are able to find pictures with their friends and family in seconds, without having to scroll through thousands of photos and videos. To ensure privacy, pictures marked by the user as “private, archived, or under review” do not appear in regular search results.

Beyond photos, PhotoPrism supports video files as well. The platform can easily play videos natively in the browser, if the relevant format is supported. If the video format is not supported, PhotoPrism can transcode the video to a compatible format, ensuring that users can view their videos seamlessly within the application.

PROTECTING PRIVACY

Photos represent some of our most cherished memories, and thus, privacy and data control are crucial aspects of any photo management app. PhotoPrism emphasizes user privacy by providing a secure and private environment for managing photos. With PhotoPrism, users have complete control over their data, as all photos are able to be stored locally. This means that users don't have to worry about the security of their photos in the cloud or the potential risks of data breaches, giving users peace of mind knowing that their data is in their own hands. This is an enticing advantage for PhotoPrism over other photo management solutions, such as cloud storage. The fragility of cloud storage has been a major concern for years, with some cases leading to the hacking of as much as 2.3 billion files and 11 million photos. Not to mention, given the rise of photo manipulation and the growing prevalence of deepfakes, many are seeking safer ways to store personal photos.

Storage of photos through PhotoPrism is a safe and simple process. Docker is the recommended tool for running PhotoPrism which allows it to run on Mac, Linux, and Windows all with few commands.

Beyond safety concerns, local storage can add convenience to the process and eliminate the intimidating and time-consuming process of uploading photos to the ever-mysterious cloud. Mayer explains, “Many people think it’s more convenient to have cloud application, but if you have a lot of pictures, it’s not very convenient to upload all of them and can take days or weeks. It’s much more convenient to manage them locally.”

THE TECH STACK

PhotoPrism is built using a robust tech stack that enables its powerful photo management capabilities.

Central to the backend of PhotoPrism is Go, a Google-developed programming language known for its speed and efficiency. Go allows for fast processing and handling of large amounts of data, making it ideal on the backend of a photo management program.

As mentioned before, metadata extraction is key to the inter-workings of the platform. The application utilizes pre-trained AI models, such as ImageNet, for image classification and recognition. The image-recognition model helps in automatically detecting and labeling objects, people, and locations in photos. Additionally, PhotoPrism extracts metadata, such as time, location, and file names, from the image files to enhance organization and search capabilities. This data can also be accessed by the users. According to the PhotoPrism website, supported fields in the metadata include: “Title, Description, Date, Location, Camera, Lens, ISO, F Number, Exposure, Focal Length, Subject, Artist, Keywords, and Copyright.”

The image-recognition and data extraction system is threshold-based, making for smoother image detection and more search-friendly results. “For example,” Mayer explains, “Cat pictures will be very reliably displayed … but generic things like windows or doors, even if there is one in the scene, might get ignored at a higher threshold.” Mayer and Gresch are the first to implement this threshold-based, search-friendly system into photo management, and their GitHub published model has since been used to power other open source, image detection and image search projects.

Although PhotoPrism is building off this solid and proprietary technological foundation, the team continues to look for ways to improve. Challenges and ongoing plans for improving AI capabilities in PhotoPrism include exploring more advanced models and incorporating more detailed object detection for a finer-grained analysis of images.

Simplifying installation and expanding user accessibility are also key focuses for the team. Currently, minor technical skills are required to install and set up the application. However, the team aims to develop a more user-friendly installation process and possibly a mobile app as well, making it accessible to a wider range of users. This will involve creating a binary that can be easily downloaded and installed.

A COMMUNITY EFFORT

The PhotoPrism community has grown significantly since its launch, with an estimated 40,000 active instances of the application from all over the world. The project has gained popularity on GitHub, where it is recognized as one of the top projects in the AI space! The growing popularity of the project indicates the trust and support of the developer community in the underlying infrastructure and emphasizes the need for a more secure and interactive photo management solution.

Engagement with the community is a priority for the PhotoPrism team, and the open source structure of their model allows for contributions to the project roadmap from anyone passionate about the project. This promotes transparency and community collaboration amongst a data-sensitive project.

A NEW ERA OF PHOTO MANAGEMENT

Are the days of struggling to organize photos and store them securely over? Thanks to this new AI innovation, it seems so. This AI-driven solution is ushering in a new wave of powerful and privacy-centric photo management. It offers nuanced features, efficient organization, and complete control over your photo library.

Time to sign off … I have some selfies to organize!

WANT TO SEE THE FULL INTERVIEW?

Our FryAI team sat down with Michael Mayer and Theresa Gresch, founders of PhotoPrism, to discuss the project as well as the capabilities of AI in photo management. You can view the full conversation here: