Machine Learning Scientist -- Expert with Bigquery

Job Description We are looking for a strong AI Data Engineer / MLOps Engineer to join our team full-time on a long-term basis. This role is ideal for someone who can work across data engineering, AI/LLM model deployment, model evaluation, cloud architecture, and Python backend development. The ideal candidate should have hands-on experience building production systems that collect, process, store, evaluate, and serve AI/ML model outputs. You should be comfortable designing cloud-based architectures, working with structured and unstructured data, building Python APIs, and creating evaluation pipelines for AI and LLM-based systems. This is a full-time role requiring availability from 9 AM to 5 PM Pacific Time, Monday through Friday. Please do not apply if you cannot consistently work during these hours. Responsibilities AI / LLM Model Deployment and Evaluation Build and maintain evaluation pipelines for AI/ML and LLM-based systems Design workflows for testing model performance, accuracy, reliability, and failure cases Evaluate model outputs using metrics such as precision, recall, F1 score, false positive/false negative rates, latency, and confidence thresholds Support prompt engineering, prompt testing, and LLM response evaluation Build structured evaluation datasets and validation workflows Apply post-processing logic to model outputs, including filtering, thresholding, smoothing, aggregation, and rule-based corrections Track model versions, prompt versions, data versions, and performance over time Help deploy AI and LLM systems into production environments Data Engineering and Analytics Design and build data pipelines for collecting, processing, and storing AI/model outputs Work with structured logs, model predictions, human review labels, evaluation results, and operational data Design clean database schemas and analytics-ready tables Build data workflows using BigQuery, SQL, Python, and cloud services Create pipelines for batch and real-time data ingestion Build dashboards, reports, and internal tools for tracking model and system performance Ensure data quality, consistency, validation, and traceability across pipelines Cloud Architecture and Infrastructure Design scalable cloud architectures using GCP services Work with services such as: BigQuery Cloud Run Cloud Storage Pub/Sub Cloud SQL Vertex AI Cloud Functions IAM Cloud Logging and Monitoring Architect systems for data ingestion, processing, model evaluation, model serving, and backend APIs Help design reliable, secure, and scalable cloud workflows Support deployment, monitoring, and debugging of production AI services Python Backend Development Build backend services and APIs using Python Develop REST APIs using frameworks such as FastAPI, Flask, or Django Design APIs for receiving model predictions, serving model outputs, storing evaluation results, and integrating with dashboards Write clean, maintainable, production-quality Python code Implement logging, validation, error handling, authentication, retries, and testing Work with backend engineers, AI engineers, and product teams to integrate AI systems into production products Required Skills Strong proficiency in Python Strong understanding of AI/ML model evaluation Experience with LLMs, prompt engineering, and LLM output evaluation Experience deploying or integrating AI/ML/LLM systems into production Strong experience with data engineering concepts: Data pipelines ETL/ELT SQL Data validation Data modeling Batch and/or streaming workflows Hands-on experience with GCP, especially BigQuery Experience designing cloud-based architectures using managed cloud services Experience building backend APIs using Python frameworks such as FastAPI, Flask, or Django Experience working with REST APIs, databases, cloud storage, and production logs Ability to debug data, model, backend, and cloud issues end-to-end Strong communication skills and ability to work closely with engineering teams Strong Requirement Must be available 9 AM to 5 PM Pacific Time Must be available 40 hours per week Must be able to communicate clearly during PST business hours Must be comfortable working in a fast-moving startup environment Nice to Have Experience with Vertex AI model deployment Experience with MLflow, Weights & Biases, Langfuse, or similar tools Experience with RAG pipelines, embeddings, vector databases, or AI agents Experience with Docker and CI/CD pipelines Experience with Cloud Run, Pub/Sub, Cloud Functions, or Cloud SQL Experience with model monitoring and observability Experience designing evaluation datasets for computer vision, NLP, or LLM systems Experience with healthcare, IoT, edge AI, or real-time data systems Ideal Candidate The ideal candidate is not just a data engineer, backend engineer, or ML engineer — they can connect all three areas. You should be able to: Understand AI and LLM model behavior Design evaluation pipelines to measure model quality Build data pipelines to collect and analyze model outputs Design cloud architecture using GCP services Build Python backend services and APIs Help move AI systems from prototype to production This role is a strong fit for someone who enjoys working at the intersection of AI engineering, data engineering, cloud architecture, and backend development.