Data Engineer II

• Design, build, and maintain scalable batch and real-time data pipelines using Maxwell, Kafka, Spark, and dbt to power analytics and business-critical applications.
• Develop and optimize data models following Medallion Architecture (Bronze, Silver, Gold) to create reliable, reusable, and high-quality datasets.
• Build and maintain cloud-native data platforms using S3, Spark, Trino, and BigQuery, ensuring scalability, reliability, and cost efficiency.
• Design robust data ingestion frameworks leveraging CDC (Maxwell), Kafka, and event-driven architectures to support near real-time data processing.
• Create, optimize, and maintain data warehouses and data marts that enable fast, reliable reporting and self-service analytics.
• Partner closely with Product Managers, Data Analysts, Backend Engineers, and Business stakeholders to translate business requirements into scalable data solutions.
• Develop reusable dbt models, testing frameworks, and documentation to improve data quality, governance, and developer productivity.
• Optimize Spark jobs, Trino queries, and storage layouts for performance, reliability, and cost efficiency.
• Own the end-to-end lifecycle of critical data pipelines, ensuring high availability, monitoring, SLA adherence, and proactive incident resolution.
• Build and enhance the core data platform by developing reusable frameworks, automation, CI/CD pipelines, and engineering best practices.
• Ensure data quality through validation, monitoring, lineage, and observability while implementing best practices for security and governance.
• Enable analytics teams by delivering trusted datasets, semantic models, and dashboards that power decision-making through Metabase.

Requirements

• 4+ years of hands-on experience designing and building scalable data platforms, data lakes, and data warehouses.
• Strong proficiency in Spark (Scala, python) and SQL, with experience building production-grade data pipelines and distributed data processing applications.
• Hands-on experience with Apache Spark and a solid understanding of distributed data processing, performance tuning, and optimization.
• Experience building batch and streaming data pipelines using technologies such as Kafka, CDC/Maxwell, or similar event-driven architectures.
• Strong understanding of modern data lake architectures, including Medallion Architecture, data modeling, partitioning, and storage optimization.
• Experience working with cloud-native data platforms and technologies such as Amazon S3, BigQuery, Trino, or similar analytics engines.
• Solid experience designing dimensional models, star schemas, and building reliable data marts that support analytics and business intelligence.
• Hands-on experience with dbt, including developing reusable models, implementing automated testing, and maintaining documentation.
• Strong knowledge of data quality, observability, lineage, and engineering best practices to build reliable and maintainable data products.
• Experience optimizing large-scale data pipelines, SQL queries, and distributed processing jobs for performance, scalability, and cost efficiency.
• Familiarity with CI/CD, Git-based development workflows, infrastructure automation, and modern software engineering best practices.
• Excellent problem-solving skills with the ability to independently own projects from design through production.
• Strong communication and stakeholder management skills, with experience collaborating across Product, Engineering, Analytics, and Business teams.
• A passion for building scalable data platforms and continuously improving developer experience, platform reliability, and operational excellence.

Benefits What We Offer You Inclusive and Diverse Environment: We foster an inclusive and diverse workplace that values innovation and offers remote environments. Competitive Compensation: Our compensation packages are highly competitive and include potential share options for certain roles. Personal Growth and Development: We are committed to your personal and professional growth, providing regular training and an annual learning stipend to help you advance your career in a dynamic environment. Autonomy and Mentorship: You'll enjoy a high degree of autonomy in your role, supported by mentorship and ambitious goals that pave the way for both your success and the company's growth.