Job description
Job details
- Location: Barkingside, Ilford
- Work mode: Remote
- Employment type: Full-time (Not an internship)
- Salary: GBP 45,000 - 85,000 per year
Role overview
N Consulting Ltd is seeking a Senior Data Engineer to lead the design and implementation of scalable data pipelines. This is a remote full-time position focused on leveraging Databricks and AWS to transform complex datasets into actionable insights using a Medallion architecture.
Job details
This role is based in Barkingside, Ilford, but operates as a Remote position. This is a Full-time employment opportunity and is not an Internship. The salary range for this position is £45,000 - £85,000 GBP per year.
Responsibilities
- Develop and maintain robust ETL processes to move and transform data across Bronze, Silver, and Gold layers.
- Design and implement scalable data models to support business intelligence and analytics requirements.
- Write high-performance PySpark and Python code for complex data transformations within Databricks.
- Collaborate with stakeholders to define data architecture and pipeline requirements.
- Optimize SQL queries and data storage strategies to improve system performance and reliability.
Requirements
- At least 7 years of professional experience in data engineering or a similar technical role.
- Hands-on expertise with Databricks development and AWS cloud infrastructure.
- Strong proficiency in Python, PySpark, and advanced SQL development.
- Proven experience implementing Medallion architecture using ADF and PySpark.
- Deep understanding of data modelling principles and ETL pipeline orchestration.
- Excellent communication skills for collaborating with cross-functional teams.
Benefits
- Competitive annual salary between £45,000 and £85,000 GBP.
- Fully remote work flexibility.
- Opportunity to work with a modern cloud data stack.
- Professional growth in a consulting environment.
Keywords
DatabricksAWSPythonPySparkSQLETLAzure Data FactoryData Modelling