Job type
Full-time
Location
Poland - Warsaw
Experience
Senior
Category
Data Engineering
Posted
4 days ago
Apply by
No deadline
We are looking for a Senior Data Engineer to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture. This role is focused on Data Products development. The successful candidate will build pipelines in ADF, DAGs in Airflow and work closely with POs to refine business requirements and transform business needs into reliable Data Products in Databricks. We are looking for a Senior Data Engineer to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture. This role is focused on Data Products development. The successful candidate will build pipelines in ADF, DAGs in Airflow and work closely with POs to refine business requirements and transform business needs into reliable Data Products in Databricks. Key Responsibilites Hands-on Data Product development Implement scalable, reliable and reusable data pipelines using Azure Data Factory and Apache Airflow. Develop data transformations in Azure Databricks (PySpark / SQL) to create/extend data products following the Medallion Architecture layers: Bronze (raw ingestion) Silver (cleansed & conformed) Gold (business-ready data products) Optimize performance, cost, and reliability of data workloads. Raise important questions during business maturation in order to detect risks before development stage. Technical Leadership Participate in architecture reviews and technical decision-making. Promote engineering excellence through code reviews and design reviews. Collaborate with Architects, Product Owners in data product technical maturation. Support & Operations (Run Team Collaboration) Provide support to the Run/Operations team for production pipelines and data products for complex topics. Troubleshoot and resolve data incidents, failures, and performance issues Participate in incident management, root cause analysis, and post-mortems. Ensure proper handover, documentation, and operational readiness of developed solutions. Contribute to improving runbooks, monitoring, and operational processes.
Requirements
Azure Data FactoryApache AirflowDatabricksPySparkSQL
Market rate — Data Engineering, Poland · based on 1 similar listings, last 90 days
This job is hosted by Margo Group. Clicking Apply opens their site.
Your Match
See how well your skills line up with this role, and what you're missing.
AI Cover Letter
Generate a cover letter tailored to this job from your profile.