About the mission
our organization is seeking a Data Engineer to develop and maintain a data and information platform in Databricks on Microsoft Azure, utilizing Power BI for reporting. The role involves analyzing and modeling data analytics use cases, collaborating with stakeholders, and ensuring operational functionality of the platform.
Read the full description
Overview
our organization is building a data and information platform in Databricks on a Microsoft Azure Cloud environment with Power BI as the reporting tool. Various sources and applications are integrated into this platform, which is built in different data and information layers to create (advanced) analytics use cases and reports. our organization is currently organized into 4 different domain teams and one foundational team. You will join the domain team that addresses the needs of the Network Management Directorate. In this project, we are looking for a Data Engineer to:Further develop this platform according to our organization's architecture and design principles.
Analyze, model, and shape use cases regarding data analytics and business intelligence with the various components on the data and information platform.
Maintain the our organization data and information platform in operational functionality.
We expect people from the 'data' world who have a data mindset, live and breathe data, but also possess the necessary IT skills. This project is approached in a project-based manner under the leadership of our organization and in collaboration with our organization colleagues.Key Responsibilities
Collaborate on functional analyses/specifications and discuss them with various stakeholders (information analysts, information stewards, key users, ...).
Based on needs, work out the data model (in the different layers) according to the rules in the platform.
Be responsible for the technical implementation in Databricks and use the framework provided by the data foundational team.
Follow the standard development cycle within the platform.
Assist in validating the setup using a test plan and help provide test data to execute test cases.
Ensure that your work fits within our monitoring and alerting setup so that all errors are immediately visible to meet agreed service levels.
Keep documentation up to date (in Confluence and Jira).
Assist power users and key users in the business with resolving technical issues.
Skills & Experience
Proven experience as a BI Consultant
Proven experience in Data Analysis
Proven experience in data warehouse modeling
Proven experience in data warehousing concepts, processes, and architectures
Proven experience with CI/CD pipelines / DevOps
Proven experience with Data Management (ETL)
Proven experience with Data Vault Modeling
Proven experience with Databricks
Proven experience with and knowledge of Python (or PySpark) programming language
Proven experience with Power BI (semantic model, governance, and reporting)
Proven experience with SQL
Languages
Language requirement: Dutch at European CEFR level C2.
Additional Info
Proven Asset Management domain knowledge is a significant advantage
Proven experience as a data engineer in an MS Azure environment (Azure Data Factory, Azure Key Vault, Azure DevOps, Azure Storage Accounts)
Proven experience in the Belgian energy sector
Proven experience with Agile/scrum working
Proven experience with file formats (.xml, .json, .parquet, .cpkg) and databases (Oracle, SQL Server, ...)
Proven experience with reporting tools
Proven experience with requirements analysis
Proven experience with Star modeling
Databricks Certified Data Engineer Professional
Proven experience in data warehouse security aspects
Proven experience in SAP BW / SAP Datasphere / SAP Business Data Cloud
Proven experience with DAX
People Management
Based on their expertise, this person must be able to fully integrate into the existing team within 2 weeks. Self-organization and a strong learning ability are essential qualities.
You are communicative, maintain personal contacts, and build a good relationship with the user community.
You are a true team player and actively participate in project work.
Innovation is one of your strengths where you are constantly looking for the best solution.
You are analytical, work structured, and document your work.
What you'll do
- Collaborate on functional analyses/specifications and discuss them with various stakeholders (information analysts, information stewards, key users, ...)
- Based on needs, work out the data model (in the different layers) according to the rules in the platform.
- Be responsible for the technical implementation in Databricks and use the framework provided by the data foundational team.
- Follow the standard development cycle within the platform.
- Assist in validating the setup using a test plan and help provide test data to execute test cases.
- Ensure that your work fits within our monitoring and alerting setup so that all errors are immediately visible to meet agreed service levels.
- Keep documentation up to date (in Confluence and Jira).
- Assist power users and key users in the business with resolving technical issues.
What we're looking for
- Proven experience as a BI Consultant
- Proven experience in Data Analysis
- Proven experience in data warehouse modeling
- Proven experience in data warehousing concepts, processes, and architectures
- Proven experience with CI/CD pipelines / DevOps
- Proven experience with Data Management (ETL)
- Proven experience with Data Vault Modeling
- Proven experience with Databricks
- Proven experience with and knowledge of Python (or PySpark) programming language
- Proven experience with Power BI (semantic model, governance, and reporting)
- Proven experience with SQL
- Language requirement: Dutch at European CEFR level C2
Skills
- Databricks
- Microsoft Azure
- Power BI
- Python
- SQL