Lead Data Engineer - Scala/Spark

Job Title : Lead Data Engineer - Scala/Spark

Job Location : Bengaluru

Exp Range : 6-12 years

Position Overview : We are seeking a Senior Data Engineer with deep expertise in Scala-based Spark development and end-to-end deployment of data pipelines on Kubernetes cluster, orchestrated via Airflow.

Key Responsibilities:

• Design & implement robust, scalable, batch & real-time data engineering solutions using Apache

Spark (Scala) & Spark structure streaming.

• Architect well-structured Scala projects using reusable, modular, and testable codebases aligned with SOLID principles and clean architecture principles & practices.

• Develop, Deploy & Manage Spark jobs on Kubernetes clusters, ensuring eTicient resource utilization, fault tolerance, and scalability.

• Orchestrate data workflows using Apache Airflow — manage DAGs, task dependencies, retries,...