Policy Chatbot with Azure AI Search and GPT-4, for Teams & Copilot Studio

Written By Nandit Dhania

This blog post details how I built a smart chatbot that provides instant access to company policy information. By combining the power of Azure Blob Storage, Azure AI Search with text embeddings, and GPT-4, I created a solution that delivers accurate, context-aware answers to employee policy questions, enhancing efficiency and improving access to critical information.

Why these resources are the best fit

Building a robust and intelligent policy chatbot requires a combination of storage, search, and conversational AI capabilities.

Azure Blob Storage

  • Scalable and Cost-Effective: Blob Storage efficiently handles large volumes of policy documents, scaling as needed while keeping costs under control.

  • Secure and Reliable: Provides secure and reliable storage for sensitive policy information, ensuring data integrity and availability.

  • Easy Integration: Seamlessly integrates with other Azure services, simplifying the development and deployment process.

  • Use Case Relevance: Serves as the central repository for all policy documents, providing a single source of truth for the chatbot.

Azure AI Search with Text Embeddings

  • Semantic Search: Text embeddings enable semantic search, allowing the chatbot to understand the intent behind user queries and retrieve relevant information even if the wording doesn't exactly match the policy documents.

  • Improved Relevance: Delivers more accurate and relevant search results compared to traditional keyword-based search.

  • Enhanced Search Experience: Provides a more intuitive and user-friendly search experience, making it easier for employees to find the information they need.

  • Use Case Relevance: Ensures that the chatbot can accurately interpret user questions and retrieve the most relevant policy information.

GPT-4

  • Natural Language Understanding: GPT-4's advanced language understanding capabilities enable it to interpret complex user queries and extract the underlying intent.

  • Conversational AI: Facilitates natural and engaging conversations, providing users with a user-friendly experience.

  • Context-Aware Responses: Generates contextually relevant and informative responses, ensuring that users receive accurate and helpful information.

  • Use Case Relevance: Powers the chatbot's conversational interface, providing employees with clear and concise answers to their policy questions.

By combining these powerful Azure resources, a smart policy chatbot can empower employees to quickly and easily access the information they need, improving efficiency and promoting a culture of informed decision-making.

Step-by-step guide

Setting Up the Foundation: Resource Group Creation

Before diving into the technical components, it's essential to organize your Azure resources effectively. A resource group acts as a container for related resources, simplifying management and deployment.

Steps:

  • Log in to the Azure portal.

  • Navigate to "Resource groups" and click "Create."

  • Provide a name for your resource group (e.g., "PolicyChatbotRG").

  • Select the appropriate region for your resources.

  • Click "Review + create" and then "Create" to create the resource group.

Storing Policy Documents: Azure Blob Storage

  • In your resource group, click "Create" and search for "Storage account." You can search using the search bar too.

  • Select "Storage account" and click "Create."

  • Provide a name for your storage account (e.g., "policydocumentsstore").

  • Choose the appropriate performance tier (e.g., Standard).

  • Click on upload and create a new container for your documents.

  • Upload all your files to the container.

Leveraging AI: Azure OpenAI and Embeddings

Azure OpenAI provides access to powerful embedding models that are crucial for vectorizing the policy documents and enabling semantic search.

Steps:

Search for "Azure OpenAI”, select and click "Create." Follow the prompts to create your Azure OpenAI resource. You'll need to select a pricing tier .

  • Once your Open AI resource is deployed, click on “Go to Azure AI Foundry portal”.

  • Under “Deployments", select “Deploy model” and then select, “Deploy base model”.

  • Select the model (e.g., text-embedding-ada-002) and click on confirm.

  • Keep the Deployment type to Standard and deploy.

Powering the Search: Azure AI Search

Azure AI Search will index the vectorized policy documents and provide the search functionality for the chatbot.

Steps:

  • Search for "Azure AI Search."

  • Select "Azure AI Search" and click "Create."

  • Provide a name for your search service (e.g., "policy-search").

  • Choose the appropriate pricing tier.

  • Select "Review + create" and then "Create" to create the search service.

  • Once created, click on “Import and vectorise data”. This will vectorise the policy documents using the OpenAI embeddings model and import the vector data into the Azure AI Search index.

  • Select “Azure Blob Storage”.

  • Select the subscription, the storage account we created, and the container.

  • Select the embeddings model to vectorise the text.

  • That’s all the configuration we need for this, click on Create.

  • Under “Indexes”, you can now see your index has been created.

  • Let’s give it a try in the “Search explorer”, see how responses look and how they are scored.

Integrating GPT-4 with Azure AI Search

This section outlines how to deploy and configure GPT-4 to interact with your Azure AI Search index, enabling the conversational capabilities of your policy chatbot.

  • Deploy a chat completion model, I am using GPT-4-32K for this

  • Update the deployment name if you need to, select “Standard” deployment type and click on “Create resource and deploy’.

  • Once the model is deployed, select the model and open Playground.

  • Add the prompt to the model. Here’s an example:

    “You are a friendly and conversational AI assistant designed to help employees quickly access company policy information. You use an Azure AI Search index containing vectorized policy documents as your primary data source. When a user asks a question, understand their intent and provide relevant information from the index in a conversational and helpful manner. If the user's request is unclear, ask clarifying questions to ensure you understand their needs. Your knowledge base includes policies related to IT equipment requests, social media usage, leave allowances, and career breaks. Focus on providing concise and accurate answers, and always cite the specific policy document where the information is found.”

  • Click on “Add your data” to add the Azure AI Search index. Select the AI Search service and the index.

  • Select semantic search type and select your semantic search configuration.

  • Click on Save and close, and you’re done!

With your model deployed and Azure AI Search configured, you're ready to put your smart policy chatbot to the test! Start interacting with it and experience how it provides friendly, conversational responses to user queries, complete with source citations for enhanced transparency and trust.

Consider deploying it as a custom web application for broader accessibility, embed it within Microsoft Teams for enhanced team collaboration, or leverage Copilot Studio to create a conversational experience directly within Microsoft 365 applications. Choosing the right deployment strategy will maximize the chatbot's reach and impact within your organization.

Nandit Dhania