Senior Data Engineer

This role is for one of the Weekday's clients

Min Experience: 5 years

Location: Bengaluru

JobType: full-time

We are looking for a seasoned Senior Data Engineer to design, build, and maintain scalable data pipelines that support growing data volumes and complex processing needs. This role involves close collaboration with cross-functional teams to enhance data models that power business intelligence tools, with the goal of improving data accessibility and enabling data-driven decision-making across the organization.

Requirements

Key Responsibilities:

Data Pipeline Development and Maintenance:

• Design, develop, and manage robust data pipelines and ETL workflows on AWS.
• Utilize AWS services including S3, Glue, Lambda, and RDS to handle and process large-scale datasets.
• Support analytics and machine learning use cases with efficient and scalable data solutions.

DevOps and Automation:

• Apply DevOps best practices in data engineering workflows, with a focus on automation, CI/CD, and infrastructure as code (IaC).
• Develop and manage IaC templates using tools like Terraform or CloudFormation to provision AWS infrastructure.
• Implement monitoring, logging, and alerting systems to ensure the reliability, performance, and quality of data pipelines and models.

Qualifications:

• Minimum of 5 years of hands-on experience in building and managing ETL/ELT processes.

Key Skills & Competencies:

• Cloud Technologies: Proficient with AWS services such as S3, Glue, Lambda, RDS, and SageMaker.
• Programming: Advanced knowledge of Python and experience with libraries like Pandas for data processing.
• DevOps: Familiarity with CI/CD pipelines, Terraform/CloudFormation for infrastructure automation, and monitoring tools like CloudWatch.
• Data Engineering: Strong grasp of data warehousing concepts and ETL/ELT pipelines.
• Collaboration: Excellent communication and interpersonal skills; ability to work effectively in cross-functional Agile teams.
• Problem Solving: Strong analytical thinking and a solution-oriented mindset.
• Soft Skills: Effective planning, time management, organizational abilities, and the capacity to work under tight deadlines.

Skills:
Python, AWS, AWS Lambda, AWS Glue, AWS S3, Pandas, DevOps, ETL, ELT, RDS, PostgreSQL, SQL, CI/CD