Revolutionizing user experience with Gen-AI powered chatbot

Introduction:  

Fibe has consistently aimed at serving young professionals, a group often overlooked by traditional financial institutions. Since its inception, Fibe has processed over a million loans and has been pivotal in simplifying access to financial services, making it a beloved tool for mid-income individuals across India. 

Fibe, hereafter referred to as “Customer”. 

Industry: Fintech 

Offering: Gen AI

Business Need:  

Customer was using traditional dialogue driven chatbot to answer the loan related queries. The queries range from simple FAQ to customer specific questions related to payment due dates etc. Because the customer was using dialogue driven approach, lots of questions from end users were restricted to receive answers only for predefined questions. This caused an increase in the number of calls to their call center which in turn increased the operational cost. The customer was looking for a context based free flow chatbot to interact with the end users.   

Implementation:  

To address the identified shortcomings and enhance the chatbot’s capabilities, a GenAI-based solution is proposed. This solution will integrate several key features to ensure improved performance and adaptability: 

• Knowledge Base Management: Develop a centralized system to manage FAQ sheets, APIs, and other business documents, streamlining information access and updates. 

• Contextually Relevant Responses: Utilize advanced LLMs to generate tailored responses based on user queries, enhancing accuracy and usefulness. 

• Free-flow Human-like Conversation: Leverage existing data for the chatbot to engage in natural, personalized conversations with users, improving user experience 

• Integrate RAG System: Incorporate a Retrieval Augmentation & Generation system to dynamically retrieve real-time data, enriching chatbot responses with up-to-date information. 

• Handling Dynamic and Static Queries: Equip the chatbot to effectively handle both real-time and static inquiries, ensuring timely and accurate responses to user queries. 

AWS Deployment Architecture: 

Implementation Strategies and Enhancements: 

      1. Chat Pipeline: 

• Platform Evolution: 

• • The chatbot journey commenced with Amazon Bedrock Claude 2 and evolved to Amazon Bedrock Claude Instant 2.1 for expedited response times. 

• Contextual Enrichment: 

• • Amazon Bedrock Titan was instrumental in embedding the FAQ sheet, augmenting responses with contextual understanding.

• Data Management and Authentication: 

• • Amazon DynamoDB facilitated data storage for session management and contextualization of user queries. 

• • The chatbot adeptly managed two distinct user personas: guest users and registered customers, with authentication handled via backend APIs using CustomerID.

      2. Data Pipeline: 

• FAQ Management: 

• • A robust data pipeline was established to manage FAQs stored on Vector DB deployed on Amazon EKS, ensuring continuous updates without disrupting chatbot functionality. 

• Customer API Integration: 

• • Another pipeline was created to interact with Customer APIs, retrieving crucial details such as payment dates and due dates specific to a user. 

LLM-based Key Work and Improvements: 

• Model Enhancement: The switch from Claude 2.1 to Claude Instant significantly improved response times, enhancing overall user experience. 

• Parameter Tuning and Refinement: Parameter tuning and the addition of XML tags and placeholders within prompts refined response generation, leading to more accurate and contextually relevant answers. 

• Custom Logic Implementation: A bespoke logic was developed to enable agents to discern whether a query should be addressed through FAQ or necessitated a customer API call, resulting in optimized response times. 

• Admin Control Options: Administrators were provided with parameter tuning options through API headers, granting control over parameters such as Top P, token length, and temperature for tailored chatbot responses. 

• This chatbot is designed to answer queries related to fintech and based on the knowledge base vector DB. We have implemented custom guardrails approach to handle situation where end-users were asking irrelevant questions, example: How to make a lemonade?, Who is the Prime Minister of India etc.  

• We have implemented few shot prompting to make the response more appealing to the customer that ensures that response is polite and graceful and does not add statements such as “based on context”, “based on API response” in the final outcome.  

• Implemented Vectorization method while storing the data in VectorDB to ensure smooth and faster data retrieval.  

• We have also enabled ranking and filtering based on similarity threshold values and metadata stored in VectorDB. 

 AWS Services: 

1.  Amazon S3 
2. Amazon Bedrock
3. Amazon API Gateway
4. AWS Lambda
5. Amazon DynamoDB
6. Amazon EKS
7. Amazon VPC 
8. Amazon EC2
9. Amazon ECR 

Reaping Rewards:  

• With rapid response generation clocking in at an impressive 2-3 seconds, users experienced minimal wait times, significantly enhancing overall user experience. 

• Fibe chatbot is now handling more than 4 lakh conversation and over 5 Million user base, having   ~10 messages on average per conversation. This chatbot has streamlined and automated 95% of the agent based chat for dynamic queries.  

• The enhanced chatbot with its accurate responses for the end users, resulted in 15% reduction in the number of agent conversation made to the agents than before, hence contributing to saving operational cost close to 200K USD.  

• Implementation ensured better maintenance of the knowledge base and an easy-to-update pipeline, further optimizing resource utilization, and reducing operational overheads for the customer.