Cloudera(CDP) Data Engineer
Budget: -
HOURLY / FULL_TIME
⭐ 0.00 (0)
United States
hadoop, apache-nifi, sas, sap, algorithm-development, cloudera, ansible, aws-lambda, terraform, apache-impala
I am looking for energetic , and trustful, dedicated freelance person, who want to build the better career !
Responsibilities:
Design and build data pipelines to extract, transform, and load (ETL) large data sets from multiple sources into the Cloudera environment.
Manage and optimize data infrastructure for high performance, reliability, and scalability across both on-premise and cloud environments.
Develop and maintain scripts and workflows using Python, Java, Scala, or Pig to automate data processing and integration tasks.
Collaborate with cross-functional teams to understand data requirements, develop solutions, and ensure data is accurate and available for analytics and reporting.
Monitor and troubleshoot Cloudera environments, leveraging tools such as Cloudera Manager and Hue for system health, tuning, and debugging.
Qualifications
3+ years of experience using Hadoop technologies (including Spark) to ingest, transform, and process data
Experience with Cloudera installation, configuration, tuning, and administration
Experience developing and managing NiFi pipelines for data ingestion and transformation
Experience leveraging generative AI coding tools to assist in development
Experience working in SQL (Hive, Spark SQL, or Impala) for querying and managing data within the Cloudera ecosystem
Experience with public cloud platforms such as AWS or Microsoft Azure
Knowledge of Python, Java, Scala, or Bash for data engineering and automation
Experience with Terraform for infrastructure automation and deployment
Experience with with CI/CD tools and DevOps best practices
Knowledge of data governance, metadata management, and data catalog tools
Ability to optimize queries and resource usage for better performance and efficiency
Öppna på Upwork