Data Engineer
Full job description
Propeller-Tech
Salary: £360 – £480 per day
Data Engineer
Location: Option of: Offices in London, Reading, Bristol, Cardiff (Remote Opportunities too)
Contract Position
Our client is a leading provider of cloud-based software and technology solutions to increase productivity for legal and business professionals. In essence, they provide critical information, services and workflows used by clients to manage their process, information and regulatory requirements.
The Role
As a Data Engineer you will work with Product, Commercial, Operations and back-office teams to build and operate next generation strategic platforms and products through innovation with data. A deep understanding of data architecture, engineering, analysis, and reporting is required.
You may be deployed into one of several cross-functional product teams to architect, design and build cutting-edge solutions in a fast-paced environment.
You will be joining at an exciting period as they continue their journey into a serverless first cloud native architecture underpinned by strong automation capabilities, messaging patterns, automated testing, and a vibrant learning culture.
Your input and experience will further help to shape the future technical roadmap, best practices, and culture.
Key responsibilities
- Design, implement and support innovative scalable Data Engineering solutions
- Support the introduction and support of data streaming platforms into the technical ecosystem.
- Champion data quality and automation
- Assemble large, complex data sets to meet functional / non-functional business requirements
- Design, build and implement ETL/ELT processes
- Produce and maintain technical and design documentation
- Support introduction and support of distributed data storage and processing platforms.
- Promote engineering excellence, security best practices, and industry standards
What are we looking for?
Ideally you will have a Bachelor’s degree in computer science, Engineering, or a related field; or demonstrable equivalent experience and training/personal
development.
- Bachelor’s degree in computer science, Engineering, or a related field; or demonstrable equivalent experience and training/personal development
- Production experience with one of the main cloud providers (preferably AWS or Azure)
- Must have experience in supporting Dremio
- Software development experience with distributed data storage and processing technologies as Hadoop, Spark, R and Python.
- Experience in leading the architecture, implementation, delivery, and operationalisation of ETL/ELT data processing pipelines on both cloud and on-prem environments.
- Experience in designing, delivering, and maintaining data mesh architectures
- Experience in configuring workflows as code using software engineering practices as modularity, testing and CI/CD
- Experience in working with Azure DevOps, GitHub, or SVN tools
- Experience with event driven architectures and data steaming using technologies and platforms as Kafka and CDC products.
- Strong data modelling experience with relational, document, graph, log stores and other non-relational platforms.
- Experience with large and complex structured and unstructured data.
- Leadership experience in implementing and operating teams using Scrum and Kanban.
- Working with DevOps principles, Git, IaC release process, automated CI/CD pipelines, etc
- Coaching and mentoring experience in data development disciplines
To apply for this job please visit definitejobs.co.uk.