The Pic2Recipe breakthrough in Machine Learning has taught the computer how to cook!
Researchers from MIT have built upon the Food-101 dataset which was an effort in 2014 designed to teach the computer how to identify different foods in photos. Now that the computer can identify different foods, the next step is to identify the recipe.
Check out the quick video below to get a feel for how the computer looks at an image of some food, then offers a suggestion for a recipe on how to cook that dish.
The algorithm is new and still needs work, but the results are already pretty impressive from their demonstration website.
http://tuesday.csail.mit.edu:4242/
I uploaded an image of spaghetti first and it had no idea how to make the dish. The research team indicated that since the Food-101 dataset contained a very large number of deserts, the system seems to be best at predicting the recipe for cookies and muffins.
So next I uploaded the following royalty free image...
The results that came back this time were as follows...
Result # 1 with 80% Certainty
1 cup quick oats
1/4 cup raisins
1/4 teaspoon salt
1/2 teaspoon pumpkin pie spice
1/3 cup canned pumpkin
1 tablespoon vegetable oil
1/2 teaspoon vanilla extract
2 tablespoons honey
Combine all the dry ingredients.
Mix in the wet ingredients and roll into balls.
Serve or refrigerate them for later consumption.
Result # 2 with 79% Certainty
2 cups whole wheat flour
2 tablespoons whole wheat flour
1/2 teaspoon salt
1/2 teaspoon ground cinnamon
1 teaspoon baking powder
1 18 teaspoons baking soda
1 cup light butter, melted
2 large eggs (I like egglands best)
78 cup honey
1 1/2 tablespoons molasses
1 tablespoon pure vanilla extract
3 tablespoons milk
3 cups old fashioned oats
1 cup raisins
1/4-1/2 cup raw chopped nuts (optional)
Mix all ingredients until well combined- except oats, raisins and nuts.
Mix in Oats, raisins, and nuts.
Mix well.
Drop onto lightly greased baking sheet using ice cream scoop.
Bake at 325 degrees for 18-20 minutes.