Data Engineer - BaKhabar Kissan

Job Description

Responsibilities

• Collect, clean, and process data from various SQL data sources, ensuring accuracy, completeness, and consistency.
• Perform data validation and quality checks to identify and resolve anomalies, errors, and inconsistencies in data sets.
• Develop and maintain data management systems, including data dictionaries, metadata repositories, and data documentation.
• Plan, deploy and maintain data pipelines and ETL (Extract, Transform, Load) processes (Python/NodeJS) on Linux based servers to move and transform data from various source databases (MySQL/PostgreSQL) into the target systems.
• Design and develop data architectures, databases, and data processing systems to support the collection, storage, and retrieval of structured and unstructured data.
• Conduct data profiling, analysis, and validation to identify data anomalies, patterns, and trends, providing insights for business decisio...