Data Engineer

Please note even though the GPP mentions Remote, this is a Hybrid role.

Key Responsibilities:

Implement and automate deployment of distributed systems for ingesting and transforming data from various sources (relational, event-based, unstructured).

Continuously monitor and troubleshoot data quality and integrity issues.

Implement data governance processes and methods for managing metadata, access, and retention for internal and external users.

Develop reliable, efficient, scalable, and quality data pipelines with monitoring and alert mechanisms using ETL/ELT tools or scripting languages.

Develop physical data models and implement data storage architectures as per design guidelines.

Analyze complex data elements and systems, data flow, dependencies, and relationships to contribute to conceptual, physical, and logical data models.

Participate in testing and troubleshooting of data pipelines.

Develop and operate large-scale data storage and processing solutions using distributed and cloud-based platforms (e.g., Data Lakes, Hadoop, Hbase, Cassandra, MongoDB, Accumulo, DynamoDB).

Use agile development technologies, such as DevOps, Scrum, Kanban, and continuous improvement cycles, for data-driven applications.

RESPONSIBILITIES

Qualifications:

College, university, or equivalent degree in a relevant technical discipline, or relevant equivalent experience required.

This position may require licensing for compliance with export controls or sanctions regulations.

Competencies:

System Requirements Engineering:

Translate stakeholder needs into verifiable requirements and establish acceptance criteria.

Collaborates:

Build partnerships and work collaboratively with others to meet shared objectives.

Communicates Effectively:

Develop and deliver multi-mode communications that convey a clear understanding of the unique needs of different audiences.

Customer Focus:

Build strong customer relationships and deliver customer-centric solutions.

Decision Quality:

Make good and timely decisions that keep the organization moving forward.

Data Extraction:

Perform ETL activities from various sources and transform them for consumption by downstream applications and users.

Programming:

Create, write, and test computer code, test scripts, and build scripts using industry standards and tools.

Quality Assurance Metrics:

Apply measurement science to assess whether a solution meets its intended outcomes.

Solution Documentation:

Document information and solutions based on knowledge gained during product development activities.

Solution Validation Testing:

Validate configuration item changes or solutions using best practices.

Data Quality:

Identify, understand, and correct flaws in data to support effective information governance.

Problem Solving:

Solve problems using systematic analysis processes and industry-standard methodologies.

Values Differences:

Recognize the value that different perspectives and cultures bring to an organization.

QUALIFICATIONS

Skills and Experience Needed:

Must-Have:

3-5 years of experience in data engineering with a strong background in Azure Databricks and Scala/Python.

Hands-on experience with Spark (Scala/PySpark) and SQL.

Experience with SPARK Streaming, SPARK Internals, and Query Optimization.

Proficiency in Azure Cloud Services.

Agile Development experience.

Unit Testing of ETL.

Experience creating ETL pipelines with ML model integration.

Knowledge of Big Data storage strategies (optimization and performance).

Critical problem-solving skills.

Basic understanding of Data Models (SQL/NoSQL) including Delta Lake or Lakehouse.

Quick learner.

Nice-to-Have:

Understanding of the ML lifecycle.

Exposure to Big Data open source technologies.

Experience with SPARK, Scala/Java, Map-Reduce, Hive, Hbase, and Kafka.

SQL query language proficiency.

Experience with clustered compute cloud-based implementations.

Familiarity with developing applications requiring large file movement for a cloud-based environment.

Exposure to Agile software development.

Experience building analytical solutions.

Exposure to IoT technology.

Work Schedule:

Most of the work will be with stakeholders in the US, with an overlap of 2-3 hours during EST hours on a need basis.

Job

Systems/Information Technology

Organization

Cummins Inc.

Role Category: Remote

Job Type: Exempt - Experienced

ReqID: 2409179

Relocation Package

Yes