Retrieval-Augmented Conversational Recommendation with Prompt-based Semi-Structured Natural Language State Tracking

Abstract.

Conversational recommendation (ConvRec) systems must understand rich and diverse natural language (NL) expressions of user preferences and intents, often communicated in an indirect manner (e.g., “I’m watching my weight”). Such complex utterances make retrieving relevant items challenging, especially if only using often incomplete or out-of-date metadata. Fortunately, many domains feature rich item reviews that cover standard metadata categories and offer complex opinions that might match a user’s interests (e.g., “classy joint for a date”). However, only recently have large language models (LLMs) let us unlock the commonsense connections between user preference utterances and complex language in user-generated reviews. Further, LLMs enable novel paradigms for semi-structured dialogue state tracking, complex intent and preference understanding, and generating recommendations, explanations, and question answers. We thus introduce a novel technology RA-Rec, a Retrieval-Augmented, LLM-driven dialogue state tracking system for ConvRec, showcased with a video, open source GitHub repository, and interactive Google Colab notebook.

1. Introduction

Effective conversational recommendation (ConvRec) systems need to understand rich and diverse natural language (NL) expressions of user preferences and intents, often communicated in an indirect or subtle manner (Kang et al., 2017; Louis et al., 2020; Lyu et al., 2021; Hosseini et al., 2023). For instance, a user who asks “Do they have parking?” is both inquiring and providing a preference for available parking. Similarly, a user looking for a restaurant who states “I’m watching my weight” is expressing a complex preference that requires commonsense reasoning and may not match any predefined restaurant metadata fields. Metadata is also often incomplete or out-of-date, making it challenging to connect NL requests to relevant item recommendations. This creates major limitations for traditional NL ConvRec systems that rely on mapping user intents and preferences to predefined metadata taxonomies (Yan et al., 2017; Sun and Zhang, 2018; Narducci et al., 2020; Jannach et al., 2021).

Figure 1. The RA-Rec prompt-driven dialogue state tracking loop. LLM prompting is used for multi-label intent classification and for updating a JSON semi-structured NL state which tracks user preferences and other key dialogue elements. The state keys provide an easily configurable structure, while LLM-generated state values can capture nuanced NL expressions.

Table 1. User intents and system actions in RA-Rec, which are a subset of the recommendation taxonomy of Lyu et al. (Lyu et al., 2021).

User Intents Intent Description Examples Provide Preference Provide or refine preference for their desired item “I want a place with a very good scenic view.” Inquire Ask for more information about the recommended item(s) “What kind of menu do they offer?”, “How do these options compare for price?” Reject Recommendation Reject a recommended item, either explicitly or implicitly “Probably too expensive, what else is there?” Accept Recommendation Accept a recommended item, either explicitly or implicitly “The first place looks good!” System Actions Action Description Examples Request Information Request the user’s preferences towards item aspect(s) “Where are you located?”, “What kind of cuisine are you looking for?” Recommend and Explain Recommend item(s) and explain how they match user preferences “How about trying Washoku Bistro for a comfortable and laid-back vibe while enjoying some delicious Japanese sushi?” Answer Respond to user inquiry about recommended item(s) “Yes, Tokyo Express has a parking lot.” Respond to Rejection Respond to user’s rejection of recommended item(s) “I’m sorry that you did not like the recommendation. Is there anything else I can assist you with?” Respond to Acceptance Respond to user’s acceptance of recommended item(s) “Great! If you need any more assistance, feel free to ask.” Greeting Greet the user. “Hello there! I am an Edmonton restaurant recommender. How can I help you?”

Fortunately, many recommendation domains have an abundance of rich NL item reviews that not only refer to standard metadata categories but also offer more complex opinions and narratives that might match a user’s interests, e.g. “The menu had lots of low-cal veggie options!”.  However, what we have lacked until recently with the advent of large language models (LLMs) (OpenAI, 2022; Chowdhery et al., 2022; Bubeck et al., 2023) is the ability to unlock the commonsense reasoning connections between rich user preference utterances and expressive language in user-generated content such as NL reviews. In addition to bridging this language expression and reasoning gap, LLMs also provide novel opportunities to control and facilitate a range of interactions in ConvRec dialogue, such as understanding user intents and preferences, and generating recommendations, explanations, and answers to questions (Deldjoo et al., 2024).

We thus introduce a novel open source demonstration technology RA-Rec, a Retrieval-Augmented, LLM-driven dialogue state tracking system for ConvRec, making the following contributions:

• We introduce prompt-driven ConvRec intent classification and state updating that captures nuanced NL expressions while maintaining domain-specific preference structure via a semi-structured NL dialogue state (Sec. 3.2).

• We extend recent work on reviewed-item retrieval (Abdollah Pour et al., 2023) to Conv-Rec dialogue, generating state-based queries, recommendations, explanations, and question answers (Figure 2).

• We demonstrate RA-Rec for restaurant recommendation, including a video, a well-documented open source GitHub repository under a permissive MIT License, and an interactive Google Colab notebook that can run the system.

Figure 2. Retrieval-augmented recommendation and explanation using late fusion RIR. First, preferences in the dialogue state are used to generate a NL query. Then, query and review embeddings are scored using dot product similarity, and the top review scores for each item are averaged (fused) into an item score. Finally, the top items’ most relevant reviews and metadata are used to generate a recommendation and explanation of how the item satisfies the preferences in the state.

2. Background and Related Work

2.1. Dialogue State Tracking

A standard Dialogue State Tracking (DST) loop (Williams et al., 2016) has four steps: (1) intent understanding, (2) dialogue state updating, (3) action selection, and (4) response generation. A traditional state consists of keys and values, typically from a predefined set of labels such as “food: italian”, “price: cheap”, “area: east”, that represent a most likely estimate of the participants’ shared intentions and beliefs at a given turn (Cohen et al., 1987; Williams et al., 2016). State tracking techniques generally map features extracted from user utterances to state labels, and include hand-crafted rules (Larsson and Traum, 2000; Bohus and Rudnicky, 2003), discriminative classifiers (Bohus and Rudnicky, 2006) and Bayesian networks (Paek and Horvitz, 2013; Williams and Young, 2007). While following the DST loop steps for modular dialogue control, our RA-Rec system (Sec. 3) extends traditional DST methods with LLM-driven state tracking to capture complex, NL expressions of preference and to facilitate state-based retrieval-augmented recommendation and question answering (QA).

2.2. Reviewed Item Retrieval

Aiming to unlock the expressive NL data available in reviews, Abdollah Pour et al. (Abdollah Pour et al., 2023) recently extended Neural IR (Reimers and Gurevych, 2019) to an approach they call Reviewed Item Retrieval (RIR), where the key challenge lies in fusing low-level information from multiple reviews to a higher item level (Zhang and Balog, 2017). They demonstrate it is more effective to first score individual reviews against a query and then aggregate these scores to an item level (late fusion), instead of summarizing reviews at an item level before query-scoring (early fusion), since late fusion retains critical nuanced review information lost by early fusion.

In late fusion RIR, given a query and a set of reviews, a neural encoder maps each review and the query to respective embeddings. A similarity function, such as the dot product, then computes a query-review similarity score. For each item, scores from the reviews with the highest query-review similarities are then averaged (fused) to give a query-item similarity score, and the top-scoring items are returned. As we will discuss in the next section, our RA-Rec system adapts late fusion RIR to ConvRec by generating queries from a NL dialogue state and using review-based retrieval-augmented generation for recommendation and QA, as illustrated in Figure 2.

3. Retrieval Augmented Conversational Recommendation

Table 2. JSON keys in the RA-Rec semi-structured NL dialogue state. Subkeys can be modified to easily facilate new domains. Bold subkeys indicate mandatory preferences the system will request information about if these preferences are not provided.

State Key	Description	Subkeys
hard_constraints	User preferences that must be satisfied	location, cuisine_type, dish_type, price_range, atmosphere, dietary_restrictions, wait_times, type_of_meal, others
soft_constraints	User preferences that are not required
recommended_items	Previously recommended items	-
rejected_items	Previously rejected items	-
accepted_items	Previously accepted items	-

To leverage both the modular structure of a traditional DST loop and the NL reasoning abilities of LLMs, we propose RA-Rec, a modular, retrieval-augmented ConvRec system, illustrated in Figure 1. We employ a prompt-driven approach for intent classification and state updating, with the latter relying on a JSON format NL state that can be configured with domain-specific keys while capturing nuance through LLM-generated NL values. We then use this NL state to facilitate personalized, retrieval-augmented recommendation and QA utilizing item reviews and metadata.

3.1. Prompt-Driven Intent Classification

After the user makes an utterance, LLM-prompting is used to determine whether the user expresses any of the four intents in Table 1, which are a subset of the recommendation dialogue intent taxonomy of Lyu et al. (Lyu et al., 2021). Table 3 outlines the prompts used in RA-Rec, with full prompt templates available in the GitHub repository (see Sec. 1). We take a multi-label intent classification approach to capture multiple intents that might be expressed in a single utterance — for example, the utterance “Does Washoku Bistro have parking?” should be classified using both the intents “Inquire” and “Provide preference” because it expresses a preference towards available parking. A larger set of user intents can be facilitated by updating the system’s prompts and initial state keys.

3.2. Semi-Structured NL Dialogue State Tracking

We store descriptions of user preferences and other important conversational elements such as rejected recommendations in a JSON state using the keys shown in Table 2 — two state examples are in Figures 1 and 2. While the keys provide structure, the state values are typically LLM-generated from the latest utterances, allowing the state to represent complex NL expressions of preference such as “I’m watching my weight” at a level of nuance and expressivity that would be impossible with predefined value sets. We thus refer to this state representation as a semi-structured NL dialogue state.

3.2.1. State Elements

Since the goal of RA-Rec is recommendation, the most important components of the state maintain an up-to-date belief about user preferences, represented through hard (required) and soft (not required) constraints. In our restaurant recommendation demo, these constraints are represented with several domain-specific subkeys listed in Table 2, as well as an “others” subkey to capture any unspecified preference types. To adapt RA-Rec to a new domain, these restaurant-specific subkeys can be replaced with domain-specific subkeys with little effort from a system designer.

Other state elements include previously recommended, rejected, or accepted restaurants – more elements could be easily added to handle a wider set of (domain-specific) user intents and system actions. Most state values are LLM-generated (prompts are summarized in Table 3) and used downstream for action selection, recommendation, explanation, and QA, discussed next.

Table 3. The main prompts used in RA-Rec – full templates can be found in the repository documentation (see Sec. 1 link).

Component	Prompt	Description
Intent Classification	Classify Intent	Given a user utterance and description of an intent (e.g. inquire), identify whether the utterance expresses the intent.
State Update	Update Constraints	Given a user utterance and the previous hard and soft constraints, update the constraints.
	Update Accepted/Rejected Item	Given a user utterance with intent ”Accept/Reject Recommendation”, identify which item was accepted/rejected.
Recommendation and Explanation	Generate Recommendation Query	Given hard/soft constraints in the state, generate a NL query.
	Explain Recommendations	Given the top retrieved items, their metadata, and their top reviews, explain how these recommended items match the hard/soft constraints.
QA	Determine QA Knowledge Source	Given a user inquiry about recommended items and those items’ metadata, identify which fields should be used to answer the inquiry, if any. If none, reviews will be used as the QA knowledge source.
	Answer Using Metadata	Given an inquiry and relevant metadata entries, generate an answer.
	Generate QA Query	Given a user inquiry utterance, generate a NL query.
	Answer Using Reviews	Given an inquiry and retrieved reviews, generate an answer.

3.3. Action Selection

The main system actions, summarized in Table 1 are Request Information, Recommend and Explain, and Answer. To understand our Request Information implementation, consider a user asking for a restaurant recommendation without giving a location preference — a recommendation may yield a restaurant in the wrong city! To avoid such premature recommendations with insufficient context, we identify mandatory preferences that the system must ask before recommending if not already provided by the user. In our demo, mandatory preferences are location and cuisine_type as shown in Table 2, but this selection is easily customized. Once mandatory preferences have been provided, the system will Answer if the user has made an inquiry and Recommend and Explain otherwise.

3.4. Retrieval-Augmented Recommendation and Explanation

To leverage expressive user review content in RA-Rec, we provide a novel adaptation of retrieval-augmented generation (Lewis et al., 2020) for late fusion recommendation and explanation. To do this, we first generate a query based on semi-structured preferences in the dialogue state and then retrieve relevant items using RIR (Sec. 2.2) over both the item reviews and known metadata. This process is illustrated in Figure 2 with relevant prompts summarized in Table 3.

Specifically, after a NL query is generated from the hard and soft constraints in the state, we implement late fusion RIR to retrieve a list of top-$k$ scoring items. Our implementation of late fusion RIR uses a TAS-B dense encoder (Hofstätter et al., 2021) (a variant of BERT (Devlin et al., 2019) fine-tuned for retrieval), dot product similarity, and approximate maximum-inner product search (MIPS) via FAISS (Douze et al., 2024) to enable scalability to large review corpora. After the top-$k$ items ($k=2$ in our demo) are retrieved, we use the metadata and top-scoring reviews for each item in a prompt to generate a recommendation and explanation of how these items match the dialogue state preferences.

3.5. Retrieval-Augmented Question Answering

As observed by Lyu et al. (Lyu et al., 2021), the later stages of a recommendation conversational often involve a number of inquiries about the recommended item to confirm that it meets the user’s requirements. To address such QA, RA-Rec retrieves relevant reviews or metadata for each of the items in question and uses this retrieved information to generate an answer – with Table 3 outlining the prompts used in our QA approach. Our framework is capable of addressing both individual item questions such as “What kind of menu do they offer?” as well as comparative questions such as “How do their prices compare?” as demonstrated in the video (see Sec. 1).

In more detail, the first step of QA uses prompting to determine whether an inquiry can be answered using available metadata, which is typically the best knowledge source for simple questions about common properties. In our restaurant recommendation demo, such common metadata fields include price, delivery availability, and parking information. If the inquiry cannot be answered with metadata, a NL query is generated from the user utterance and used to retrieve several reviews for each item in question. As discussed above, reviews are an expressive knowledge source, especially when inquiries and preferences are stated in complex ways. Finally, the retrieved reviews and metadata for each item are used to generate an answer to the question, which may include item comparisons.

3.6. RA-Rec System Summary

In summary, RA-Rec employs an LLM-driven, modular DST structure to facilitate a controllable recommendation dialogue that can connect complex NL user preferences to matching items using their reviews and metadata. Its JSON semi-structured NL state features configurable keys for domain-specific control while the LLM-updated state values are able to express NL nuance. This state supports novel retrieval-augmented recommendation, explanation, and QA, using scalable retrieval methods such as late fusion RIR and leveraging item reviews and metadata to generate responses.

4. Demonstration Details

Our system is designed for easy adaptation to various domains, and as a demonstration, we present RA-Rec for restaurant recommendation — see Sec. 1 for demo links. Specifically, we use the Yelp Academic Dataset to obtain metadata and over 46K reviews for 1298 restaurants in Edmonton, Alberta. GPT-3.5-turbo is the LLM used for all prompting steps, but the RA-Rec framework is LLM-agnostic and will work with any prompt-based LLM model.

5. Future Work

RA-Rec is a flexible LLM prompt-driven architecture and thus opens many new directions for ConvRec systems to support natural user workflows (Lyu et al., 2021; Jannach et al., 2021). Key extensions include support for (1) active preference elicitation to narrow down large item spaces (Austin et al., 2024), (2) structured reasoning over multi-aspect NL preferences (Zhang et al., 2023), and (3) trade-off negotiation between multiple recommendations.