Line OA Chat Analytics & AI Grading System

1. System Requirements & General Function The goal of this project is to build an automated background script that analyzes monthly chat histories directly from our hotel's LINE Official Account, evaluates how well our staff is doing, and turns that data into a clean PDF report for management. Chat Logs: The system needs to extract the chat conversations from our LINE Official Account and separate guest messages from employee responses. Business Hours Only: The system needs to track reply timestamps but exclude any after-hours or out-of-office windows from its speed calculations so the final data stays fair. Automatic Delivery: The script must run completely on its own every month, using an email API or SMTP setup to send the finished PDF report straight to the management team. 2. Report Structure & Required Sections The generated PDF document needs to look clean, professional, and easy for managers to read quickly. Each employee's report must include the following sections: A. Basic Overview Details: The employee’s name and the specific month being reviewed. Volume: The total number of separate chat interactions handled during that period. B. AI Performance Grading (Scale 1–10) The system will use an LLM API (like Gemini, OpenAI, or Claude) to score the employee based on three main criteria: Politeness: How well the employee uses proper hospitality greetings, polite language, and parting phrases. Language Tone: An evaluation of how helpful, clear, and welcoming the responses feel. Professional Alignment: How well the employee follows brand guidelines and handles complaints without needing to escalate the issue. C. Response Time Analytics Speed Metrics: Calculations showing the employee's average response time and peak response time during normal business hours. Data Chart: A clean, embedded bar chart (histogram). The bottom axis (X-axis) will show response time blocks (like 0–5 minutes, 5–10 minutes), and the side axis (Y-axis) will show how many chats fell into those blocks. D. AI Written Feedback Box A dynamic text box filled in by the AI API that gives specific narrative notes: Core Strengths: Bullet points highlighting what the employee did well, such as great customer service or consistent tone. Areas for Improvement: Clear notes pointing out where the employee was too slow or used language that was too casual, along with tips on how managers can coach them. 3. User Acceptance Testing (UAT) The 2-Week Test: Before the project is officially approved and finished, we require a strict 2-week testing period to make sure the script is completely stable. Testing Rules: During these 14 days, the script will run on a fast schedule. It must successfully create and email an error-free report every 2 days (14 total reports) without any system crashes, API failures, or broken formatting.