Data Engineer - Senior

GPP Database Link (https://cummins365.sharepoint.com/sites/CS38534/)

Job Summary:

Leads projects for the design, development, and maintenance of a data and analytics platform. Effectively and efficiently processes, stores, and makes data available to analysts and other consumers. Works with key business stakeholders, IT experts, and subject-matter experts to plan, design, and deliver optimal analytics and data science solutions. Works on one or many product teams at a time. Though the role category is generally listed as Remote, this specific position is designated as Hybrid.

Key Responsibilities:

Designs and automates deployment of distributed systems for ingesting and transforming data from various sources (relational, event-based, unstructured).

Designs and implements frameworks to continuously monitor and troubleshoot data quality and integrity issues.

Implements data governance processes, including metadata management, access control, and retention policies for internal and external users.

Provides guidance on building reliable, efficient, scalable, and quality data pipelines with monitoring and alert mechanisms that combine a variety of sources using ETL/ELT tools or scripting languages.

Designs and implements physical data models to define database structures and optimize database performance through indexing and table relationships.

Participates in optimizing, testing, and troubleshooting data pipelines.

Develops and operates large-scale data storage and processing solutions using distributed and cloud-based platforms (e.g., Data Lakes, Hadoop, HBase, Cassandra, MongoDB, Accumulo, DynamoDB).

Uses innovative tools, techniques, and architectures to automate common data preparation and integration tasks, minimizing manual and error-prone processes.

Assists in renovating data management infrastructure to drive automation in data integration and management.

Ensures the timeliness and success of critical analytics initiatives by using agile development methodologies such as DevOps, Scrum, and Kanban.

Coaches and develops less experienced team members.

RESPONSIBILITIES

Competencies:

System Requirements Engineering:

Translates stakeholder needs into verifiable requirements, tracks status, and assesses impact changes.

Collaborates:

Builds partnerships and works collaboratively with others to meet shared objectives.

Communicates Effectively:

Delivers multi-mode communications tailored to different audiences.

Customer Focus:

Builds strong customer relationships and provides customer-centric solutions.

Decision Quality:

Makes good and timely decisions that drive the organization forward.

Data Extraction:

Performs ETL activities from various sources using appropriate tools and technologies.

Programming:

Develops, tests, and maintains code using industry standards, version control, and automation tools.

Quality Assurance Metrics:

Measures and assesses solution effectiveness using IT Operating Model (ITOM) standards.

Solution Documentation:

Documents knowledge gained and communicates solutions for improved productivity.

Solution Validation Testing:

Validates configurations and solutions to meet customer requirements using SDLC best practices.

Data Quality:

Identifies, corrects, and manages data flaws to support effective governance and decision-making.

Problem Solving:

Uses systematic analysis to determine root causes and implement robust solutions.

Values Differences:

Recognizes and leverages the value of diverse perspectives and cultures.

QUALIFICATIONS

referred Experience:

Intermediate experience in a relevant discipline is required.

Knowledge of the latest technologies and trends in data engineering is highly preferred.

Familiarity with analyzing complex business systems, industry requirements, and data regulations.

Background in processing and managing large datasets.

Design and development experience for Big Data platforms using open-source and third-party tools.

Experience with SPARK, Scala/Java, Map-Reduce, Hive, HBase, and Kafka.

Hands-on experience with SQL query language.

Experience in clustered compute cloud-based implementations.

Experience developing applications requiring large file movement in a cloud-based environment and using various data extraction tools.

Experience in building analytical solutions.

Exposure to IoT technology.

Experience in Agile software development methodologies.

Technical Skills:

Programming Languages:

Proficiency in Python, Java, and/or Scala.

Database Management:

Expertise in SQL and NoSQL databases.

Big Data Technologies:

Hands-on experience with Hadoop, Spark, Kafka, and similar frameworks.

Cloud Services:

Experience with Azure, Databricks, and AWS platforms.

ETL Processes:

Strong understanding of Extract, Transform, Load (ETL) processes.

Data Replication:

Working knowledge of replication technologies like Qlik Replicate is a plus.

API Integration:

Experience working with APIs to consume data from ERP and CRM systems.

Education, Licenses, and Certifications:

Bachelor's degree in a relevant technical discipline, or equivalent experience required.

This position may require licensing for compliance with export controls or sanctions regulations.

Job

Systems/Information Technology

Organization

Cummins Inc.

Role Category: Remote

Job Type: Exempt - Experienced

ReqID: 2410679

Relocation Package

No