STM Journals Peer-reviewed research from Consortium e-Learning Network

Journal of Artificial Intelligence Research & Advances Review Article

Recipe-Fusion: Multimodal Food Recipe Recommendation System

  1. Karthik Anilkumar Nair Department of Computer Engineering, Pillai College of Engineering, New Panvel
  2. Sanjog Gajanan Chavhan Department of Computer Engineering, Pillai College of Engineering, New Panvel
  3. Tejas Nilkanth Pawar Department of Computer Engineering, Pillai College of Engineering, New Panvel
  4. Anmol Jagdish Reddy Department of Computer Engineering, Pillai College of Engineering, New Panvel
  5. K.S. Charumathi Department of Computer Engineering, Pillai College of Engineering, New Panvel

Abstract

The food recipe recommendation system using data science is a software solution designed to help users discover new and delicious food options based on their food history and other relevant data. This system recommends various recipes based on the input given by the user and it helps to filter out the recipes on course type, diet type, and nature of the food (including non-veg, and veg) using a recommendation technique. The system also analyzes food trends and popular dishes that the users might not have considered before. The system can recognize the input ingredients provided by the user and give the recipe based on the ingredients mentioned. Various machine learning and deep learning approaches like convolutional neural networks (CNN) and recurrent neural networks (RNN) will be used for the implementation of this system. The CNN model is trained and implemented to recommend the recipes based on the image similarity score. The RNN model is trained and implemented for the prediction of the recipes from the input (recipe name) given by the user. Both models (CNN and RNN) are used simultaneously for the implementation of the system, which is why it is named as "Multimodal Food Recipe Recommendation System". This system will help the person learn and make recipes based on the input given to it. This system will simplify finding relevant recipes for the user.

Keywords

References (25)

  1. Patel R, Thakkar P, Ukani V. CNNRec: Convolutional Neural Network based recommender systems - A survey. Engineering Applications of Artificial Intelligence. 2024;133:108062. doi:10.1016/j.engappai.2024.108062
  2. Forouzandeh S, Rostami M, Berahmand K, Sheikhpour R. Hfrs-Han: Health-Aware Food Recommendation System Based on the Heterogeneous Attention Network. 2023. doi:10.2139/ssrn.4378052
  3. Khilji AFUR, Sinha U, Singh P, Ali A, Dadure P, Manna R, et al. Multimodal Recipe Recommendation System Using Deep Learning and Rule-Based Approach. SN Computer Science. 2023;4(4). doi:10.1007/s42979-023-01870-6
  4. Castiglia G, El Majjodi A, Starke AD, Narducci F, Deldjoo Y, Calò F. Nudging towards health in a conversational food recommender system using multi-modal interactions and nutrition labels. 4th edn of Knowledge-aware and Conversational Recommender Systems (KaRS) Workshop @ RecSys, Seattle, USA, September 2023.
  5. Zhang J, Li M, Liu W, Lauria S, Liu X. Many-objective optimization meets recommendation systems: A food recommendation scenario. Neurocomputing. 2022;503:109-117. doi:10.1016/j. 2022.06.081.
  6. Jia N, Chen J, Wang R. An attention-based convolutional neural network for recipe recommendation. Expert Systems with Applications. 2022;201:116979. doi:10.1016/j.eswa.2022.116979
  7. Lei Z, Ul Haq A, Zeb A, Suzauddola M, Zhang D. Is the suggested food your desired?: Multi-modal recipe recommendation with demand-based knowledge graph. Expert Systems with Applications. 2021;186:115708. doi:10.1016/j.eswa.2021.115708
  8. Chavan P, Thoms B, Isaacs J. A Recommender System for Healthy Food Choices: Building a Hybrid Model for Recipe Recommendations using Big Data Sets. Proceedings of the Annual Hawaii International Conference on System Sciences. 2021. doi:10.24251/hicss.2021.458
  9. Manna R, Das D, Gelbukh A. Question-Answering and Recommendation System on Cooking Recipes. Computación y Sistemas. 2021;25(1). doi:10.13053/cys-25-1-3899
  10. Wang W, Duan LY, Jiang H, Jing P, Song X, Nie L. Market2Dish: Health-aware Food Recommendation. ACM Transactions on Multimedia Computing, Communications, and Applications. 2021;17(1):1-19. doi:10.1145/3418211
  11. Pham HX, Guerrero R, Pavlovic V, Li J. CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence. 2021;35(3):2423-2430. doi:10.1609/aaai.v35i3.16343
  12. Deldjoo Y, Schedl M, Hidasi B, Wei Y, He X. Multimedia recommender systems: Algorithms and challenges. In: Recommender Systems Handbook. Springer: New York, USA; 2021. p. 973-1014.
  13. Marin J, Biswas A, Ofli F, Hynes N, Salvador A, Aytar Y, Weber I, Torralba A. Recipe1M+: A dataset for learning cross-modal embeddings for cooking recipes and food images. arXiv Preprint arXiv:1810.06553. 2018 Oct 14.
  14. Xie Z, Liu L, Wu Y, Li L, Zhong L. Learning TFIDF Enhanced Joint Embedding for Recipe-Image Cross-Modal Retrieval Service. IEEE Transactions on Services Computing. 2022;15(6):3304-3316. doi:10.1109/tsc.2021.3098834
  15. Tian Y, Zhang C, Metoyer R, Chawla NV. Recipe Representation Learning with Networks. Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021:1824-1833. doi:10.1145/3459637.3482468
  16. Chen Y, Zhou D, Li L, Han JM. Multimodal encoders for food-oriented cross-modal retrieval. In: Web and Big Data. Proc 5th Int Joint Conf, APWeb-WAIM 2021, Guangzhou, China, 23–25 Aug 2021, Vol 5. Springer International Publishing; 2021. p. 253-266.
  17. Pan L, Chen J, Liu S, Ngo CW, Kan MY, Chua TS. A Hybrid Approach for Detecting Prerequisite Relations in Multi-Modal Food Recipes. IEEE Transactions on Multimedia. 2021;23:4491-4501. doi:10.1109/tmm.2020.3042706
  18. Pecune F, Callebert L, Marsella S. A recommender system for healthy and personalized recipes recommendations. In: HealthRecSys@ RecSys. 2020 Sep 26;15-20.
  19. Rostami A, Pandey V, Nag N, Wang V, Jain R. Personal food model. In: RT-PA of the 28th ACM Int Conf Multimedia. 2020 Oct 12;4416-4424.
  20. Theodoridis T, Solachidis V, Dimitropoulos K, Daras P. A Cross-Modal Variational Framework For Food Image Analysis. 2020 IEEE International Conference on Image Processing (ICIP). 2020:3244-3248. doi:10.1109/icip40778.2020.9190758
  21. Twomey N, Fain M, Ponikar A, Sarraf N. Towards Multi-Language Recipe Personalisation and Recommendation. Fourteenth ACM Conference on Recommender Systems. 2020:708-713. doi:10.1145/3383313.3418478
  22. Lin AS, et al. A recipe for creating multimodal aligned datasets for sequential tasks. arXiv Preprint arXiv:2005.09606. 2020.
  23. Liu A, Yuan S, Zhang C, Luo C, Liao Y, Bai K, et al. Multi-Level Multimodal Transformer Network for Multimodal Recipe Comprehension. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020:1781-1784. doi:10.1145/3397271.3401247
  24. Meng L, Feng F, He X, Gao X, Chua TS. Heterogeneous Fusion of Semantic and Collaborative Information for Visually-Aware Food Recommendation. Proceedings of the 28th ACM International Conference on Multimedia. 2020:3460-3468. doi:10.1145/3394171.3413598
  25. Fu H, Wu R, Liu C, Sun J. MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Model. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020:14558-14568. doi:10.1109/cvpr42600.2020.01458
Support