L

Senior Data Engineer - Databricks (Enterprise Data Platform Architect)

Lumenalta Canada
Remote
Apply
AI Summary

Lead the design and implementation of scalable enterprise data platforms using Databricks and AWS. Combine software engineering expertise with data platform architecture to build robust pipelines, enforce best practices, and drive cloud-native data solutions. Collaborate cross-functionally to ensure maintainability, security, and governance in distributed environments.

Key Highlights
Architect and implement modern enterprise data platforms using Databricks, Delta Lake, and Unity Catalog
Lead end-to-end data platform implementations with hands-on experience in PySpark, Python, and SQL
Drive DevOps practices including CI/CD, Infrastructure as Code (Terraform), and version control
Key Responsibilities
Design and implement scalable enterprise data platforms using Databricks and AWS cloud services
Architect and oversee end-to-end data platform implementations with a focus on Delta Lake and Unity Catalog
Develop and maintain robust data pipelines using Python, PySpark, and SQL while adhering to software engineering best practices
Implement and manage Databricks platform components including Delta Lake, Unity Catalog, Databricks Asset Bundles, and Lakehouse Jobs
Collaborate with engineering teams to ensure scalable, secure, and maintainable data solutions
Apply Infrastructure as Code (Terraform) to support repeatable and reproducible environments
Contribute to architectural decisions and guide best practices for the data platform
Technical Skills Required
Databricks Python AWS
Benefits & Perks
Flexible working hours in a remote environment
Health insurance (medical and dental) for T4 Employees
Professional development fund
Nice to Have
Experience designing lakehouse architectures at scale
Optimization of distributed data workloads
Experience with AWS Glue (ETL, jobs, crawlers, workflows)
Governance, security, and data lineage framework implementation

Job Description


Role Overview

We are seeking a Senior Data Engineer - Databricks with strong expertise in Databricks to help design and implement modern enterprise data platforms. This role blends strong software engineering practices with advanced data platform architecture, requiring hands-on experience building scalable pipelines and designing robust data systems in the cloud. The engineer will contribute to platform architecture, implement reliable data solutions, and ensure best practices across development, testing, deployment, and governance within the data platform ecosystem.


Responsibilities

  • Design and implement scalable enterprise data platforms using Databricks and AWS.
  • Architect and oversee end-to-end data platform implementations.
  • Develop robust data pipelines using Python, PySpark, and SQL.
  • Apply strong engineering practices including testing, CI/CD, and version control.
  • Implement and manage Databricks platform components including Delta Lake, Unity Catalog, Databricks Asset Bundles, and Lakeflow Jobs.
  • Collaborate with engineering teams to ensure maintainable, scalable, and secure data solutions.
  • Implement Infrastructure as Code practices to support repeatable environments.
  • Contribute to architectural decisions and guide best practices for the data platform.


Requirements

  • Deep expertise in Databricks, including platform architecture and best practices.
  • Experience as a Solution Architect or Data Platform Owner designing end-to-end implementations.
  • Strong programming experience with Python, PySpark, and SQL.
  • Experience using testing frameworks such as PyTest.
  • Solid experience with Git-based workflows, CI/CD pipelines, and DevOps practices.
  • Hands-on experience with Delta Lake, Unity Catalog, Databricks Asset Bundles (DABs), and Lakeflow Jobs.
  • Experience with Infrastructure as Code using Terraform.
  • Strong AWS experience, particularly S3 and IAM roles.
  • Strong communication and collaboration skills in distributed teams.


Nice to Have

  • Experience designing lakehouse architectures at scale.
  • Experience optimizing distributed data workloads.
  • Experience working in consulting or client-facing engineering environments.
  • Experience implementing governance, security, and data lineage frameworks.
  • Strong hands-on experience with AWS Glue (ETL, jobs, crawlers, workflows).


Salary

Salary range: CA$80,000 - CA$150,000 annually, with final compensation determined by your qualifications, expertise, experience, and the role's scope.


Location:

This is a fully remote position; however, candidates must be based in regions that align with the Pacific, Central, or Eastern U.S. time zones to ensure effective collaboration with client and team schedules.


Benefits

In addition to competitive pay, we offer a variety of benefits to support your professional and personal growth, including:

  • Flexible working hours in a remote environment.
  • Health insurance (medical and dental) for T4 Employees.
  • A professional development fund to enhance your skills and knowledge.
  • 15 days of paid time off annually.
  • Access to soft-skill development courses to further your career.


Position Details

This is a full-time position requiring a minimum of 40 hours per week, Monday through Friday.


Application Deadline

Applications will be accepted until September 20th, 2026. Candidates can expect feedback by September 28th, 2026.


Similar Jobs

Explore other opportunities that match your interests

Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Not Applicable

Thomson Reuters

Canada
Visa Sponsorship Relocation Remote
Job Type Full-time
Experience Level Mid-Senior level

Lumenalta

Canada

Business Data Analyst, Cryptocurrency Exchange

Data Science
4h ago

Premium Job

Sign up is free! Login or Sign up to view full details.

•••••• •••••• ••••••
Job Type ••••••
Experience Level ••••••

gate

Hong Kong Sar

Subscribe our newsletter

New Things Will Always Update Regularly