Data Engineer – Azure

Salary: £66,000 – £72,000 per year

Data Engineer – Azure, Canary Wharf, East London, £55-60K + Bens. You must have a background as a Data Engineer ideally with experience of some of the following; ETL/ELT pipelines, SQL, Azure Data ecosystem, Azure Databricks, Data Factory, Data Lake and Synapse. Any certifications would be a plus. Strong Python or Scala, ideally PySpark would also be highly desirable.

As the Data Engineer, you will own the company’s data ecosystems and will work with developers, solution architects, technical BAs, data scientists, and other SMEs to define the optimum data architecture for the business. You will build the required ETL/ELT pipelines for the collection, preparation and storage of data in a form that is optimised, secure and reliable.

You will have a specific focus on developing the data inputs that are utilised by the organisation’s internal forecasting model; designing a framework that scales effectively along with the source data.

Responsibilities;

• Design data infrastructure to support modelling and data analytics

• Build and maintain ETL/ELT pipelines to make data accurate and easy to use

• Work to ingest and transform data sets from a variety of data sources

• Explore ways to enhance data quality and reliability

• Assist with the establishment of a data culture across the organisation

• Drive better data governance through the creation and embedding of principles and processes e.g. Logical Data Model & Flow Diagram, Data Dictionary, Data Semantic Layers

• Set service level indicators and monitor execution of data workflows and configure alerts

• Apply dimensional data modelling concepts and practices to develop conceptual, logical and physical data models to support insight delivery

• Identify data quality issues through data profiling, analysis and stakeholder engagement

Your background / experience;

• Experience of designing data infrastructure to support modelling and data analytics

• Experience building, modelling, and maintaining data pipelines

• Strong experience in SQL optimisation, performance tuning, data modelling and SQL/database design skills.

• Hands on experience within the Azure Data ecosystem, with Azure Databricks, Data Factory, Data Lake, and Synapse. Certifications are a plus.

Highly Desirable

• Strong competence in Python or Scala, ideally PySpark experience

This Data Engineer – Azure role is based in Canary Wharf and requires 2 days a week in the office.