Python & Cloud Data Engineer (GCP / Azure / BigQuery / Automation)

We are looking for an experienced Python Data Engineer / Cloud Engineer to support a data integration and analytics project. The ideal candidate should have strong hands-on experience with Python, Google Cloud Platform (GCP), BigQuery, Google Cloud Storage (GCS), Azure Cloud, API integrations, and CI/CD automation using Harness. Project Overview We need to build an automated data pipeline that: 1. Fetches user data from external APIs. 2. Cross-references the user data with team/organizational data from multiple sources. 3. Identifies missing attributes and enriches the data. 4. Uses an LLM (OpenAI, Gemini, Vertex AI, etc.) to categorize and classify records. 5. Stores the processed data in BigQuery. 6. Makes the curated dataset available for Power BI reporting and dashboard development. 7. Automates the entire workflow through cloud-native services and CI/CD pipelines. Required Skills * Strong Python development experience * REST API integration and data processing * Google Cloud Platform (GCP) * Google Cloud Storage (GCS) * BigQuery * Service Accounts & IAM * Cloud Functions / Cloud Run (preferred) * Azure Cloud experience * Data transformation and ETL/ELT pipelines * SQL and BigQuery optimization * Experience working with LLMs for data enrichment and categorization * Harness CI/CD pipeline automation * Git and Infrastructure-as-Code knowledge * Power BI data integration understanding Responsibilities * Design and develop scalable data ingestion pipelines * Integrate multiple APIs and external data sources * Build data enrichment workflows using Python and LLMs * Create automated data validation and quality checks * Load and optimize datasets in BigQuery * Configure secure cloud storage and access controls * Implement and maintain Harness deployment pipelines * Collaborate with the Power BI team to provide reporting-ready datasets * Document architecture, workflows, and deployment processes Preferred Experience * Vertex AI, OpenAI API, Gemini, or similar LLM platforms * Airflow, Cloud Composer, or workflow orchestration tools * Data warehouse design and analytics platforms * Enterprise cloud environments (AWS/GCP/Azure) * Power BI semantic models and reporting integration Engagement Details * Remote contract role * Immediate start preferred * Flexible hours, but availability for overlap meetings is required * Ongoing support and enhancements may be needed after initial delivery To Apply Please share: * Relevant GCP/Azure projects * Experience with BigQuery and GCS * Examples of API integration projects * Experience using LLMs for classification or enrichment * Harness CI/CD experience * Hourly rate and availability