Data Engineer

Job Summary:

We are seeking a talented Azure Data Engineer with strong expertise in Azure Databricks, specifically in PySpark and SQL, to join our data team. You will play a key role in building secure, scalable data pipelines and implementing robust data governance strategies.

Key Responsibilities:

• Design and implement ETL/ELT pipelines using Databricks and PySpark.
• Develop high-performance SQL queries and optimize Spark jobs.
• Collaborate with data scientists, analysts, and business stakeholders to understand data needs.
• Ensure data quality and compliance across all stages of the data lifecycle.
• Implement best practices for data security and lineage within the Databricks ecosystem.
• Participate in CI/CD, version control, and testing practices for data pipelines.

Required Skills:

• Proven experience with Databricks ...