Experience: 5–12 years
Job Description:
- Big Data Orchestration: Airflow, Spark on Kubernetes, Yarn, Oozie
- Big Data Processing: Hadoop, Kafka, Spark & Spark Structured Streaming.
- Experience on SOLID & DRY principles with Good Software Architecture & Design implementation experience
- Advanced Scala experience (e.g. Functional Programming, using Case classes, Complex Data Structures & Algorithms)
- Proficient in developing automated frameworks for unit & integration testing.
- Experience with Docker and Helm and related container technologies.
- Proficient in deploying and managing Spark workloads on Kubernetes clusters.
- Experience in evaluation and implementation of Data Validation & Data Quality
- Devops experience in Jenkins, Maven, Github, Github actions, CI/CD
Must Have exp in the following:
- (Scala + Spark + Airflow + Docker/Kubernetes)
- Data Engineer
- Scala Coding Exp
- Spark
- Airflow
- Docker
- Kubernetes
- Yarn/Oozie
- Devops