Data Engineer

This role is for one of the Weekday's clients

Min Experience: 3 years

Location: Bengaluru

JobType: full-time

We are seeking a skilled Data Engineer to design, implement, and maintain robust data infrastructure and systems. In this role, you will be responsible for building scalable and efficient data pipelines, integrating diverse data sources, and ensuring the availability and reliability of data that powers climate tech and agritech applications. Your contributions will drive the development of data-driven solutions, supporting innovation and informed decision-making across the organization.

Requirements

Key Responsibilities:

• Design, develop, and manage data pipelines and ETL processes to collect, process, and integrate data from sources such as weather data providers, satellite imagery, IoT sensors, and agricultural databases.
• Collaborate with data scientists, software engineers, and domain experts to understand data requirements and implement optimized data models and schemas.
• Optimize data storage and retrieval systems for performance and scalability, ensuring real-time or near-real-time data availability.
• Develop and enforce data quality checks and validation processes to maintain data accuracy, completeness, and integrity.
• Monitor and troubleshoot data pipelines to ensure high availability and efficient operation, proactively resolving issues or bottlenecks.
• Implement and uphold data security and privacy measures to safeguard sensitive information.
• Stay informed about emerging technologies, tools, and best practices in data engineering to continually enhance the data infrastructure.
• Create and maintain analytical reports to support cross-functional teams with daily data and reporting needs.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.
• Strong programming skills, particularly in Python.
• Proficiency in SQL and familiarity with both relational and NoSQL databases.
• Experience working with cloud platforms such as AWS, Azure, or GCP.
• Solid understanding of data modeling concepts and experience in designing and optimizing data schemas.
• Strong problem-solving skills with a high level of attention to detail.
• Excellent communication and collaboration skills for effective teamwork across departments.
• Prior experience in the agriculture domain or working with climate-related datasets is a plus.