Our client is seeking an Intermediate ETL Developer (5+years) to create data pipeline ETL Jobs using AWS Glue and PySpark within the financial services industry.
Responsibilities:
Work with a scrum team(s) to deliver product stories according to priorities set by the business and the Product Owners. Interact with stakeholders. Provide knowledge transfer to other team members. Creating and testing pipeline jobs locally using aws glue interactive session. Performance tuning of PySpark jobs. AWS Athena to perform data analysis on Lake data populated into aws glue data catalog through aws glue crawlers. Must Haves:
5+ years as an ETL Developer SQL expert AWS Glue WITH Python ( PySpark ) PySpark Dataframe API Spark SQL Knowledge in AWS services (e.g. DMS, S3, RDS, Redshift, Step Function). Nice to Haves:
Etl development experience with tools e.g.
SAP BODS, Informatica. Good understanding of version control tools like Git, GitHub, TortoiseHg. Financial services experience Agile
