Restaurant Menu Understanding: Illustrating the Need for Culturally Augmented Translation

Danilo Gallo, UX and ethnography Naver Labs Europe, France,
Jutta Willamowski, UX and Ethnography Naver Labs Europe, France,
Yada Wisatekaew, Naver Labs Europe, France,
Adrien Bruyat, Naver Labs Europe, France,
Antonietta Maria Grasso, Naver Labs Europe, France,

We carried out a user study on the issues foreign customers face when going to restaurants abroad. We describe our study and findings illustrating that simple translation is not enough to help customers understand restaurant menus in foreign countries. Similar to what is provided by human translation, where the translator mediates between the author and the reader, menu translation should augment the translated menu with complementary information to address cultural specificities and differences. It should for instance highlight local specialties, provide background information on the dishes, and explain different eating habits or manners. The information provided should bridge cultural differences and has to be adapted to the specific cultural background of the customer.

CCS Concepts:Applied computing → Language translation; • Social and professional topics → Cultural characteristics; • Information systems → Decision support systems;

Keywords: Restaurant menus, Translation, Cultural difference, User study, Food

ACM Reference Format:
Danilo Gallo, Jutta Willamowski, Yada Wisatekaew, Adrien Bruyat, and Antonietta Maria Grasso. 2020. Restaurant Menu Understanding: Illustrating the Need for Culturally Augmented Translation. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '20 Extended Abstracts), October 5–8, 2020, Oldenburg, Germany. ACM, New York, NY, USA 4 Pages.


Food is one of the main cultural attractions for travellers around the world and restaurant menus play a central role in this experience. However, understanding restaurant menus remains a challenge for foreign travellers. For menu translation, they often resort to general purpose translation tools, such as Google Translate and Papago ( However, such tools do not consider the specific restaurant and food context, and thus often provide insufficient or incorrect, out of purpose translations.

To address this issue and improve the quality of menu translation, [5] proposes to use NMT (Neural Machine Translation) trained specifically on the food domain, and producing thus a much better and more appropriate translation. Still, as we observed in our study and discuss below, foreign travellers may need more than a good translation to fully understand a restaurant menu; they may need additional, complementary information. One interesting way to augment menu translation, is to add images illustrating the dishes for the customer as proposed by [1]. In our study we find and highlight other complementary elements that should be included.

An important factor that impacts menu understanding is the cultural background of the customer. Indeed, different customers may have different issues with one and the same menu, depending on their individual cultural background. Thus, to help customers grasp a restaurant menu and the dishes proposed, it may be necessary to adapt the menu translation and complementary information provided to each specific customer's background. Providing the appropriate information for each customer requires an understanding of the gastronomy and culture of the place of origin of the food on one hand, and of the customer background on the other hand [3]. This makes automatic food translation difficult.

Existing works in the food domain that address the cultural difference issue, focus on translating individual dishes rather than menus as a whole. They describe an unknown dish to a customer based on the analysis of corresponding recipes, i.e. ingredients and cooking steps [6] and resulting dish characteristics. For instance, Nobumoto et al. [4] analyse recipe data to describe an unknown dish to a customer by citing a similar dish from her or his own culture and highlighting at the same time the main differences, again considering her or his particular culture.

In this paper we take a step back to investigate the barriers travelers face during their restaurant experience enlarging the scope beyond the individual dishes characteristics resulting from their recipe and beyond improving direct menu translation. We present a user study examining problems foreign customers have with interpreting restaurant menus. Our findings highlight the impact of cultural differences between the customer and the restaurant and the need for culturally augmented menu translation. This includes not only explaining a dish's ingredients and flavor, but also the larger culinary and cultural context, e.g. eating and social habits and manners.


We conducted a user study to understand how foreign customers access and use restaurant menus both to select a restaurant and a dish. In both cases users have to address a combination of two problems, namely selecting a restaurant or dish from a different culture, and doing this in a foreign language they do not speak. Our aim was to understand the interplay of these aspects and what makes this combination particularly challenging.

2.1 Methodology

Researchers interviewing a participant
Figure 1: Interview. We conducted the interviews in a cafe as that context can contribute in bringing the participants to the restaurant situation mindset, especially relevant during the simulation part of the study.

For our study, we carried out semi-structured interviews with non-French-speaking foreigners about their experience when visiting restaurants in a medium size French city. Each interview was conducted by one researcher with the presence of a second researcher (who observed and took notes) and was audio- and video-taped for the later analysis (Figure 1). Before starting the interview, the participants signed an agreement allowing us to collect and use the data complying with the GDPR regulations and we rewarded them with a 20€ voucher. Next, we gathered demographic data and food related information from the participants, about their interest in food and the frequency and reasons for going to restaurants. Then, our interviews were structured in two parts, the first consisting of a semi-structured interview focusing on past experiences selecting restaurants and dishes, and the second involving a simulation of the process they follow in both cases through the use of printed menus.

During the first part we asked the interviewees to describe the procedure they followed when selecting restaurants and dishes abroad, their selection criteria, the tools they used, how they accessed and used restaurant menus during the process, where the pain points were, and how language affected the selection. The goal was to confirm that restaurant menus play indeed a central role, and to understand which other factors have an impact on the process. We also asked the participants to recall and describe a recent restaurant experience, pointing out all the issues they had faced.

French restaurant menu
Figure 2: One of the 5 restaurant menus we selected for our study. This menu proposes French dishes, including in particular cheese (“fromages secs”). Cheese, well known as a typical French menu item by our participants, appears in the desert section, which contributes to making it “a risky choice” for customers.

We then moved to the second part, in which we simulated the restaurant and dish selection through the use of restaurant menus. We used the think aloud technique, asking the participants to verbalise their selection process, and to explain and show us how they use the menus for the selection. For the restaurant selection scenario, we provided the participants with a set of 5 menus from local restaurants, mainly offering French food, but also including an Asian and Italian restaurant to understand possible differences (Figure 2). Once the participants had selected a restaurant, we asked them to proceed with the dish selection from that menu. To enlarge the observation, we then asked them to do the same with the second and third preferred restaurant menus.

Participants. We recruited the participants using word of mouth, social media and flyers posted at selected locations, e.g. local institutions teaching French (Figure 3). We asked candidates to fill in a short online pre-questionnaire in which, beyond their origin and age, we asked when they had arrived in France, what their French language level was, whether they had food constraints, when their last restaurant experience was, and whether they had recently had issues during restaurant visits. The answers allowed us to select the most interesting participants, i.e. those that were rather demanding customers and more prone to experiencing problems. We finally selected 10 participants, who were between 24 and 39 years old, and originated from 8 different countries. While our initial target had been 12, we had to do a hard stop after 10 interviews due to the COVID-19 crisis. Nevertheless, we started to observe data saturation relatively quickly during our analysis.

Figure 3: Flyer inviting participants to our study and prompting them to fill an online pre-questionnaire. This pre-questionnaire helped us select the most relevant participants for our study.

Analysis. We collected over 40 minutes of video and voice recording per participant plus the notes taken by the second researcher during the interview. We followed the grounded theory approach: each recording was analyzed independently by two researchers to extract and code relevant quotes. We then gathered together as a team to structure them according to the situations where they occurred and cluster them into themes.


During our analysis we observed that there was usually a tension between the curiosity of exploring new food and taking the risk to get something that does not live up to the expectations (or even turns out to be unacceptable for the participant). Depending on the context, our participants were all more or less subject to these two opposite desires. We found that the distance between the cultures of the customer and the restaurant plays an important role: the further the customer's cultural background from the restaurant's one, the more difficult it is for the customer to evaluate the risk s/he takes.

During the study we also observed participants facing usability issues when using online tools like search engines and translators on their smart phone (Table 1). Participants also mentioned (and we observed during the simulation) their interest in finding out whether particular dishes contained specific ingredients related to dietary constraints or preferences. However, in this paper, we will not go into detail on these points and rather focus on the cultural aspects that impacted the experience, which, we believe, are more novel. Below, we discuss the various aspects of the cultural background (and the cultural difference) we observed that increase the uncertainty and in consequence the risk customers take in restaurants abroad, hindering them from making new experiences.

3.1 Flavors and ingredients

The first aspect concerns the dish itself and its characteristics in terms of flavor and ingredients. Several participants mentioned related issues. They were afraid of getting ”something too exotic” and thus searching for something ”more similar to what [they] are used to have in their country”. This aligns with findings from [2] and was in particular the case for participants coming from farther away. It happened also that participants had ordered a dish they usually find in their country but were still disappointed: ”I ordered Chawarma ... it tasted strange ... it was with beef and not with chicken as I am used to it in India.” Indeed, dishes and recipes are often adapted to the country in which they are served to suit the local taste. In this case the difference appeared between two countries having each their own different adaptation of the recipe originating from the Middle East. Providing information about the cultural adaptation of original dishes and corresponding typical ingredients and flavors such as proposed by [7] would allow to avoid such issues for the customers.

Table 1: This table shows the tools and the number of participants that use them during restaurant and dish selection. Among their issues, we observed that these tools provide limited information that is (mostly) not targeted to the restaurant context, resulting in a fragmented user experience that requires to switch manually between multiple tools. A tool integrating food related information and translation is still missing.
Tools Examples Number of
Restaurantselection Map based tools Google Maps 8
Restaurant-related tools Deliveroo, TripAdvisor 5
General purpose search Naver, Google Search 3
Dishselection Translation tools Google Translate, Word Reference 9
Dish information search Google Search 6

3.2 Dish presentation

Participants also mentioned facing issues related to the way a dish was served. They had for instance ”expected a bigger portion”, while locally that type of food was actually categorised as a side dish. They also mentioned ordering a dish with the intention to share, when they actually got ”a whole pizza for one person”. Indeed, depending on the location, pizzas can be considered both an individual or a shared dish. In another case a participant had once ordered a sandwich for lunch and was surprised that ”it was served cold and not hot as a usual meal would be” in his home country.

Participants avoided new experiences altogether when they felt too unsure about certain items on the menu. In our study, this applied in particular to cheese, well known by the participants as a typically French item, but that was too far from what they could imagine from their own culture and thus considered ”a more risky choice”. Consequently, a number of participants mentioned they had refrained from trying cheese, even if they were interested to know ”how it is consumed in France” and how it would be presented as a dish ”on the plate” (Figure 2).

All these issues around a dish's serving and presentation also relate to cultural differences. One way in which participants tried to avoid such issues was to search, before ordering, for images showing the dish, using general purpose search engines on their smartphone. However, this manual image search was too cumbersome to do for a whole menu and did also not always work out, as the returned images were sometimes not even food related. Augmenting a menu with food images as [1] propose would solve this problem. And, more generally, allowing travelers to access more accurate complementary information on their smartphone in a faster way would help them understand what to expect for each dish and facilitate the selection.

3.3 Manners

Another cultural aspect of food that came up in our interviews were different eating manners, e.g. a particular ”knife for [eating] escargots”, recalling western tourists’ experiences with chopsticks in China [2]. Manners also impacted the general behavior of participants in restaurants. For instance, they felt unsure about what they could ask to the waiter, the natural person to interact with for any request. When interested in more information or modifying the ingredients of a dish most participants refrained from asking the waiter, not to create any annoyance. As stated by one participant: “In Brazil we are very comfortable to ask for changes ... this may seem impolite here.” Providing complementary information about acceptable behaviors in specific situations according to the local culture could be a way to guide users on how to act in restaurants and to reduce the anxiety they experience.

3.4 Cultural context and specialties

Participants also mentioned being motivated by learning more about the culture of the country they were visiting. For instance, a participant appreciated how, when having dinner with a local friend, ”he was explaining [the dishes] very well, ... giving the context, ... not just some sort of translation.” Complementing the description of the dish with information about its context and history may engage travelers and encourage them to be more open to try it. This information could also be used to highlight the most traditional dishes in the menu for that type of cuisine and location.


From our findings we conclude that providing a culturally augmented menu and dish translation is required to enlarge foreign customers’ comfort zone and encourage them to try out more local specialities. As travelers usually resort to their smartphones searching for support in this kind of situations, a mobile app would be the obvious way to provide this information, making these findings relevant for those in the Mobile HCI community working on similar issues. Such a mobile app can build on and integrate existing works such as [5], [4], and [1] and complement them with other elements responding to the various needs observed in our study in order to mediate between the cultures of the user and the restaurant/location. The mobile app could thus augment the translation with the information necessary to avoid misunderstandings and generate the right expectations on the user's side about what they will get from each dish. This information would allow travelers to fully experience the gastronomic richness of each destination.


  • Arioputra Dimas and Lin Chang Hong. 2015. Mobile augmented reality as a Chinese menu translator. In 2015 IEEE International Conference on Consumer Electronics - Taiwan. 7–8.
  • Bardhi Fleura, Ostberg Jacob, and Bengtsson Anders. 2010. Negotiating cultural boundaries: Food, travel and consumer identities. Consumption Markets & Culture 13, 2 (June 2010), 133–157.
  • Liddicoat Anthony J.2016. Intercultural mediation, intercultural communication and translation. Perspectives 24, 3 (July 2016), 354–364.
  • Nobumoto Kensuke, Kato Daiju, Endo Masaki, Hirota Masaharu, and Ishikawa Hiroshi. 2017. Multilingualization of Restaurant Menu by Analogical Description. In Proceedings of the 9th Workshop on Multimedia for Cooking and Eating Activities in conjunction with The 2017 International Joint Conference on Artificial Intelligence - CEA2017. ACM Press, Melbourne, Australia, 13–18.
  • WoĹk Krzysztof. 2020. Incorporating Domain-Specific Neural Machine Translation into Augmented Reality Systems. PACIS 2020 Proceedings (June 2020).
  • Wang Liping, Li Qing, Li Na, Dong Guozhu, and Yang Yu. 2008. Substructure similarity measurement in chinese recipes. In Proceedings of the 17th international conference on World Wide Web(WWW ’08). Association for Computing Machinery, Beijing, China, 979–988.
  • Sajadmanesh Sina, Jafarzadeh Sina, Ossia Seyed Ali, Rabiee Hamid R., Haddadi Hamed, Mejova Yelena, Musolesi Mirco, Cristofaro Emiliano De, and Stringhini Gianluca. 2017. Kissing Cuisines: Exploring Worldwide Culinary Habits on the Web. In Proceedings of the 26th International Conference on World Wide Web Companion(WWW ’17 Companion). International World Wide Web Conferences Steering Committee, Perth, Australia, 1013–1021.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

MobileHCI '20 Extended Abstracts, October 05–08, 2020, Oldenburg, Germany

© 2020 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-8052-2/20/10.