Position: Data Engineer
Duration: Long Term
Location: Remote (Canada)
Job Description:
We are looking for a dedicated and detail-oriented Data Engineer to join our team. This role will work closely with the Lead Engineer and focus on strong data management practices and operational excellence aligned with our SLAs. You will play a critical role in incident investigation, response, escalation, and ensuring data quality and reliability.
Key Responsibilities :
Support in Data Disruption Investigations:
- Assist in identifying, analyzing, and resolving data disruptions quickly.
- Contribute to strategies that prevent data issues.
- Help maintain systems that detect and address data-related problems promptly.
Technical Support in Partner Interactions:
- Provide technical support during partner interactions.
- Assist in understanding and implementing technical requirements.
- Work closely with the Lead Engineer to ensure standards and expectations are met.
Data Monitoring and Escalation Support:
- Support development and implementation of data monitoring systems.
- Assist in 24/7 data observability to ensure continuous operations.
Adherence to SLAs:
- Ensure all data engineering processes follow SLA guidelines.
- Monitor and report SLA compliance and identify gaps.
Incident Investigation, Response & Escalation:
- Lead incident investigation efforts, document findings, and analyze issues.
- Develop and execute quick response strategies.
- Escalate critical incidents to relevant teams in a timely manner.
We are seeking a Senior Data Engineer with 5+ years of experience in Python and SQL. The ideal candidate should:
- Have strong programming skills and experience with enterprise data pipelines.
- Be capable of solving technical issues at scale.
- Come from a startup or smaller company environment, working with fast-paced processes and modern technologies.
- Have senior-level experience with Apache Spark and Databricks (Airflow) for processing large-scale streaming datasets.
- Have 3+ years of experience handling streaming data in AWS (Athena, Glue).
- Be familiar with CI/CD processes (preferably Terraform), SDLC best practices, design docs, code reviews, testing, etc.
- Be familiar with the modern data stack: Databricks/Snowflake, Airflow, Fivetran/Airbyte, dbt, Data Catalog, Data Lineage, Unit Testing.
- Have exposure to open-source technologies like Apache Flink, Trino, Iceberg, etc.
- Be passionate about learning new technologies and continuously improving systems.