AI system can suggest recipes based on photos

Boston: MIT scientists have developed a new artificial intelligence (AI) system that can predict ingredients and suggest recipes just by looking at photos of food.

The AI system called Pic2Recipe could help us learn recipes and better understand people’s eating habits.

“In computer vision, food is mostly neglected because we don’t have the large-scale datasets needed to make predictions,” said Yusuf Aytar, from Massachusetts Institute of Technology in the US.

“But seemingly useless photos on social media can actually provide valuable insight into health habits and dietary preferences,” said Aytar.

Researchers combed websites like All Recipes and Food.Com to develop “Recipe1M,” a database of over 1 million recipes that were annotated with information about the ingredients in a wide range of dishes.

They then used that data to train a neural network to find patterns and make connections between the food images and the corresponding ingredients and recipes.

Given a photo of a food item, Pic2Recipe could identify ingredients like flour, eggs, and butter, and then suggest several recipes that it determined to be similar to images from the database.

The system did particularly

Researchers combed websites like All Recipes and Food.Com to develop ‘Recipe1M’, a database of over 1 million recipes that were annotated with information about the ingredients in a wide range of dishes

well with desserts like cookies or muffins, since that was a main theme in the database.

However, it had difficulty determining ingredients for more ambiguous foods, like sushi rolls and smoothies. It was also often stumped when there were similar recipes for the same dishes.

For example, there are dozens of ways to make lasagna, so the team needed to make sure that system wouldn’t “penalise” recipes that are similar when trying to separate those that are different.

In the future, the team hopes to be able to improve the AI so that it can understand food in even more detail.