SENIOR DATA ENGINEER
Our Ontario based client is seeking a Senior Data Engineer to join their team.
RESPONSIBILITIES:
- Design, build and implement batch and real-time data pipelines.
- Develop and propose design patterns and ensure the proposed design, optimally addresses access and query patterns; data consumption and adheres to internal architecture standards.
- Work with various stakeholders across the business, data scientists and IT.
- Refine data requirements to meet various data and analytics initiatives and data consumption requirements.
- Increase the overall speed in which data is onboarded to the Data and Analytics platform.
- Build robust data pipelines to enable larger data consumption
- Increase the overall quality of data pipeline development through DevSecOps
TECHNICAL SKILLS :
- Programming experience in Spark using modern languages such as Python, Scala
- Experience working with modern data architectures like Azure Data Lake Storage, Azure Databricks, Azure Synapse and Delta Lakes
- Experience working with Integration patterns and technologies such as Azure Event Hub, Function App and C#
- Knowledge and expertise of database modeling techniques: Data Vaults, Star, Snowflake and 3NF
- Experience working with streaming data architecture and technologies for real-time: Spark Streaming, Kafka, Flink, Storm
- Experience working with relational and non-relational database technologies: SQL Server, Oracle, Cassandra, MongoDB, CosmosDB, HBase
- Experience working with source code and configuration management environments such as Azure DevOps, Git, Maven, Nexus
*must have Enhanced Reliability Security Clearance (MANDATORY)
1 year contract
100% Remote
Market rate based on 37.5 hours/week
If you’re interested, please reply with a copy of your updated CV.
I will be in contact quickly to discuss additional info and next steps.
I hope to hear from you soon.