Data Pipeline Development:
Design, implement, and manage scalable ETL/ELT pipelines using AWS services and Databricks.
Data Integration:
Ingest and process structured, semi-structured, and unstructured data from multiple sources into AWS Data Lake or Databricks.
Data Transformation:
Develop advanced data processing workflows using PySpark, Databricks SQL, or Scala to enable analytics and reporting.
Databricks Management:
Configure and optimize Databricks clusters, notebooks, and jobs for performance and cost efficiency.
AWS Architecture:
Design and implement solutions leveraging AWS-native services like S3, Glue, Redshift, EMR, Lambda, Kinesis, and Athena.
Collaboration:
Work closely with Data Analysts, Data Scientists, and other Engineers to understand business requirements and deliver data-driven solutions.
Performance Tuning:
Optimize data pipelines, storage, and queries for performance, scalability, and reliability.
Monitoring and Security:
Ensure data pipelines are secure, robust, and monitored using CloudWatch, Datadog, or equivalent tools.
Documentation:
Maintain clear and concise documentation for data pipelines, workflows, and architecture
Job Function: Other
Job Type: Full Time
Workplace Type: Not Specified
Experience Level: Not Specified
Salary: Competitive & Based on Experience
Experience: 0 - 0 yrs
Important Fraud Alert:
Beware of imposters. elsejob.com does not guarantee job offers or interviews in exchange for payment. Any requests for money under the guise of registration fees, refundable deposits, or similar claims are fraudulent. Please stay vigilant and report suspicious activity.
General Motors • Bengaluru, Karnataka, India
Salary: Competitive & Based on Experience
View JobRenesas • Bengaluru, Karnataka, India
Experience: 8 - 9 yrs
Salary: Competitive & Based on Experience
View JobRenesas • Bengaluru, Karnataka, India
Experience: 10 - 11 yrs
Salary: Competitive & Based on Experience
View JobGeneral Motors • Bengaluru, Karnataka, India
Experience: 10 - 15 yrs
Salary: Competitive & Based on Experience
View JobSAP Fioneer • Bengaluru, Karnataka, India
Salary: Competitive & Based on Experience
View JobAmagi • Bengaluru, Karnataka, India
Salary: Competitive & Based on Experience
View JobAxiom Software Solutions Limited • Bengaluru, Karnataka, India
Salary: Competitive & Based on Experience
View JobCommenda • Bengaluru, Karnataka, India
Salary: Competitive & Based on Experience
View JobOriental Consultants • Bengaluru, Karnataka, India
Experience: 0 - 5 yrs
Salary: Competitive & Based on Experience
View Job