Python AI Engineer (Arabic RAG Chatbot)

About the project We are a university research team building an Arabic chatbot that answers government employees’ questions using only a set of official regulations. Every answer must cite the exact article and clause it comes from. When the regulations don’t support an answer, the system must decline instead of guessing. The dataset and system design are complete, and we need help implementing and testing the system. Responsibilities Hybrid retrieval: combine BM25 (keyword search) with bge-m3 (dense embeddings), merge the results with Reciprocal Rank Fusion, then rerank them with bge-reranker-v2-m3. Include expansion through linked articles. Answer generation: integrate ALLaM-7B-Instruct (4-bit, Hugging Face Transformers) with a fixed output format that includes numbered sources and verbatim quotes. Quote verification: build a module that checks every quote against the cited article. Abstention classifier: extract the 10 pipeline features, then train a logistic regression classifier (with XGBoost for comparison) using grouped cross-validation, and select the threshold on the validation set. Evaluation pipeline: Metrics: Recall@k, MRR, answer accuracy, abstention precision/recall/F1, safe-response rate, Brier score, and bootstrap confidence intervals. Baselines: a RAG system that always answers, and a fixed reranker threshold. Experiment logging: store every intermediate result (retrieved chunks, scores, drafts, quote checks, features) so results can be reproduced. Simple interface: an Arabic, right-to-left chat interface with citations, confidence levels and like/dislike feedback. Streamlit or FastAPI with a basic front end is fine. Documentation: document the code clearly and walk the team through it, so we can explain and maintain every part. Required skills Strong Python, with experience in Hugging Face Transformers and sentence-transformers Hands-on experience building RAG systems: embeddings, hybrid search, reranking Experience running LLMs on limited GPUs: quantization with bitsandbytes, a 16 GB GPU such as the T4 scikit-learn for classification and evaluation Experience with Arabic text processing: normalization, right-to-left display, mixing Arabic and English Nice to have Experience with ALLaM, bge-m3 or other Arabic and multilingual models Knowledge of selective prediction / abstention or LLM evaluation Experience with PostgreSQL / pgvector or FAISS Experience with legal or government document systems Arabic speaker Deliverables A working retrieval module that reports Recall@5 and MRR on the validation set The full answering pipeline: retrieval, generation and quote verification A trained abstention classifier with the selected threshold Evaluation scripts that produce all metrics and baseline comparisons A simple Arabic chat interface Clean, documented code in a Git repository, plus a handover session with the team