Lead Data Engineer

**Job Description:**

We are looking for a Lead Data Engineer with strong Databricks expertise who can also design and build accelerators for automation. The role requires hands-on development in Databricks (PySpark, Spark SQL, Delta Lake,Autoloader, DLT etc) along with experience in creating frameworks for test data generation, report validation, and backend validation. You will be leading technical work, mentoring team members, and building reusable solutions that speed up project delivery.

**Key Responsibilities:**

·Design, develop, and optimize ETL/ELT pipelines on Databricks using PySpark, Spark SQL,Delta Lake, DLT live.

·Build automation accelerators such as:

Synthetic test data generators for large-scale data sets.

Automated report validation tools to compare dashboards with backend data.

Backend validation frameworks for reconciliation and data quality checks.

·Implement data validation frameworks (Great Exp...