Data Engineer – Databricks & AWS Lakehouse Solutions

Our client is seeking a skilled professional to join their data organization. This role involves designing and building scalable, reliable, and cost-efficient data pipelines on a governed lakehouse platform using Databricks on AWS.

• Design and maintain production-ready data pipelines using medallion architecture.
• Optimize ETL/ELT processes for large-scale data processing on Delta Lake.
• Implement data governance through Unity Catalog, managing access controls and data lineage.
• Build CI/CD pipelines for automated testing and deployment of data workloads.
• Configure Databricks workspaces and compute resources for optimal performance and cost.
• Collaborate with cross-functional teams in an Agile environment to translate requirements into technical solutions.

• You have experience with Python, PySpark, and SQL for data engineering.
• You possess in-depth knowledge of Databricks, Delta Lake, and Unity Catalog.
• You bring experience with AWS services including S3, IAM, KMS, and VPC.
• You're proficient in CI/CD and Infrastructure as Code using GitHub Actions, Terraform, and Databricks Asset Bundles.
• You have knowledge of medallion architecture, ETL/ELT patterns, and CDC.
• You possess the ability to work effectively in an Agile/Scrum environment.
• You are fluent in English and preferably Dutch as well 

Nice to Haves

• Experience with data quality frameworks such as Great Expectations.
• Familiarity with Lakehouse monitoring or CloudWatch dashboards.

• Start date: ASAP - long term engagement 
• Location: Gent/Zwijnaarde - hybrid (minimum 2 days/week onsite)
• Contract: open to both permanent employees and freelancers