Lead Data Engineer (Azure, Databricks, ADF)
Job Description
Description
Apex Systems is a global IT services provider and our staffing practice has an opening for a Senior Data Engineer (Azure, Databricks, ADF) . The ideal candidate will have strong experience in data engineering and Azure cloud technologies, with the ability to lead development efforts and collaborate within highly regulated environments.
Location: Downtown Toronto
Contract Duration: 6-Month Contract to Start (possibility to extend)
Key Responsibilities:
• Design and develop scalable data pipelines using Azure Data Factory (ADF), Databricks, and Azure Functions (C#)
• Develop Python notebooks and Spark workflows on Databricks for data processing and transformation
• Implement Medallion architecture for loading and organizing data from various sources
• Convert Databricks workspace from Hive Metastore to Unity Catalog and configure associated permissions
• Design and create Delta Lake schemas and tables with appropriate properties and security configurations
• Optional: Build and maintain Delta Live Tables
• Set up and manage CI/CD pipelines using Azure DevOps or GitHub Enterprise (including cluster/secret management and deployment automation)
• Work with Azure Entra ID for SSO/RBAC configuration and identity access management
• Deploy and manage Azure PaaS resources (Azure SQL, ADF, SHIR, ADLS, Synapse)
• Collaborate in a highly secure and regulated Windows environment
• Provide leadership to small teams and lead technical problem-solving sessions
Qualifications:
• 7+ years of experience in data engineering with a strong focus on Azure technologies
• Deep hands-on experience with ADF, Databricks, Azure Functions (C#), and Spark
• Strong knowledge of Python, Pandas, and notebook-based development
• Proven ability to design and implement enterprise data lake solutions
• Solid understanding of Delta Lake and Unity Catalog
• Experience with CI/CD practices and automation using Azure DevOps or GitHub Enterprise
• Strong familiarity with Azure identity and access management (SSO, RBAC)
• Experience working in regulated industries with security-compliant development processes
• Demonstrated ability to work independently and lead small teams or pods
• Excellent communication skills and a proactive mindset
• Previous experience in the financial services industry is considered an asset
Seniority level
-
Seniority level
Mid-Senior level
Employment type
-
Employment type
Contract
Job function
-
Job function
Information Technology
-
Industries
IT Services and IT Consulting and Banking
Referrals increase your chances of interviewing at Apex Systems by 2x
Get notified about new Data Engineer jobs in Toronto, Ontario, Canada .
Software Engineer, Backend (All Levels / All Teams)
New York, NY $75,000.00-$80,000.00 3 weeks ago
Data Engineer, Investment Research and Data Science
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr
Company
Apex Systems
Location
Toronto
Country
Canada
Salary
125.000
URL