Databricks Architect

Key Responsibilities

• Design and implement modern data platforms using the Databricks Lakehouse Platform.
• Build ingestion and transformation pipelines using Databricks Workflows, Apache Spark (PySpark / Spark SQL) and Delta Lake, supporting batch, streaming and near real-time processing.
• Architect focus: Define and apply platform standards for data ingestion, modelling, data quality and governance; ensure alignment to cloud, security, identity and cost management requirements.
• Develop and optimise Databricks (Apache Spark) pipelines and Delta Lake models to enable governed data consumption (for example, Power BI/Tableau) and support analytics, data science, and machine learning (ML) use cases.
• Implement monitoring, performance tuning and cluster optimisation; support CI/CD for notebooks, jobs and data pipelines and contribute to operational runbooks.
• Collaborate with analysts, data scientists and application t...