About the mission
We are looking for a Data Engineer to take technical responsibility for a critical real-time data platform in Antwerp. The role involves managing data pipelines, ensuring data quality, and working with various technologies including Elasticsearch, Databricks, and Kafka.
Read the full description
Data Engineer – Antwerp
We are looking for a Data Engineer who wants to take technical responsibility for a business-critical real-time data platform.
You will be in an environment where data plays an important role daily. You ensure that the platform remains stable, performant, scalable, and reliable, and that users can rely on accurate real-time data.
Location: Antwerp
Work regime: hybrid
Onsite: 2 days a week in Antwerp.
We are looking for a Data Engineer who wants to take technical responsibility for a business-critical real-time data platform.
You will be in an environment where data plays an important role daily. You ensure that the platform remains stable, performant, scalable, and reliable, and that users can rely on accurate real-time data.
Key Responsibilities
Take technical ownership of the Elasticsearch environment, including architecture, performance, scaling, and troubleshooting.
Manage real-time data pipelines and ensure data quality, latency, and completeness.
Proactively monitor the platform and analyze performance issues, data gaps, and production incidents.
Design and implement reliable changes, including testing, monitoring, and rollback strategies.
Further develop the real-time data architecture and APIs.
Work on integrations between Elasticsearch, Databricks, and the existing platform.
Build and productionize data workloads with Databricks and Apache Spark.
Work with streaming technologies such as Kafka and/or Azure Event Hubs.
Where necessary, work in an existing TypeScript / JavaScript / Node.js backend to analyze problems end-to-end and implement solutions.
Improve observability, scalability, and maintainability while reducing operational complexity.
Skills & Experience
You are a Data Engineer who can independently take responsibility for critical production environments.
Preferably, you bring experience with:Elasticsearch – architecture, performance, scaling, and production troubleshooting
Databricks & Apache Spark
Real-time and event-driven data architectures
Kafka and/or Azure Event Hubs
Monitoring, observability, debugging, and performance optimization
Incident analysis and failure handling
JavaScript / Node.js – sufficient knowledge to understand, debug, and modify an existing backend
Redpanda Connect / Benthos is a strong plus
Additionally, you are someone who likes to take ownership, works independently, and does not shy away from an environment where reliability and continuity are important.
Languages
You speak fluent English.
Additional Info
Location: Antwerp
Work regime: hybrid
Onsite: 2 days a week in Antwerp.
What you'll do
- Take technical ownership of the Elasticsearch environment, including architecture, performance, scaling, and troubleshooting.
- Manage real-time data pipelines and ensure data quality, latency, and completeness.
- Proactively monitor the platform and analyze performance issues, data gaps, and production incidents.
- Design and implement reliable changes, including testing, monitoring, and rollback strategies.
- Further develop the real-time data architecture and APIs.
- Work on integrations between Elasticsearch, Databricks, and the existing platform.
- Build and productionize data workloads with Databricks and Apache Spark.
- Work with streaming technologies such as Kafka and/or Azure Event Hubs.
- Where necessary, work in an existing TypeScript / JavaScript / Node.js backend to analyze problems end-to-end and implement solutions.
- Improve observability, scalability, and maintainability while reducing operational complexity.
What we're looking for
- Experience with Elasticsearch – architecture, performance, scaling and production troubleshooting
- Experience with Databricks & Apache Spark
- Experience with real-time and event-driven data architectures
- Experience with Kafka and/or Azure Event Hubs
- Experience with monitoring, observability, debugging and performance optimization
- Experience with incident analysis and failure handling
- Sufficient knowledge of JavaScript / Node.js to understand, debug and modify an existing backend
- Experience with Redpanda Connect / Benthos is a strong plus
Skills
- Elasticsearch
- Databricks
- Apache Spark
- Kafka
- Azure Event Hubs
- JavaScript
- Node.js