Pyspark

Location: PAN India Experience: 8 to 10 Years Key Skills :PySpark, Pytho n Must have Skill s: Implementing data ingestion pipelines from different types of data sources i.e Databases, S3, Files et cExperience in building ETL/ Data Warehouse transformation proce ss.Experience working with structured and unstructured da ta.Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/librari es,Developing scalable and re-usable, self-service frameworks for data ingestion and processi ng,Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of da ta,Processing performance analysis and optimizati on,Bringing best practices in following areas: Design & Analysis, Automation (Pipelining, IaC), Testing, Monitoring, Documentati on. Good to have (Knowled ge): Experience in cloud-based solut ions,Knowledge of data management princi ples.