Lead Data Engineer, Data Pipeline Architect
Tech stack - Python, SQL, ETL, PostgreSQL, Airflow, AWS, Snowflake
to £95k + shares
This is a remote position
Our client is a digital surgery platform scale-up with a rapidly-growing customer base across the UK, Europe, US and Australia.
An amazing time for a top flight Data Engineer to fast track your career and help take the data pipeline to the next level.
This is a key role within the company, mixing technical, team and product leadership, to deliver one of the company s flagship offerings to prestigious healthcare clients (including global medical device companies, state-wide US hospital networks, the NHS and large government institutions).
The Lead Data Engineer remit:
- Lead developer of the data pipeline. The warehouse (dimensionally-modelled) pulls from multiple sources at various frequencies
- Owning data quality, availability and security
- Owning the reporting interfaces including internal (currently Superset) and external (reports serving the web dashboard, and bespoke client reports)
- Mentoring the existing data engineer
- Proactively analysing and improving the quality of data products - including performance, scalability, maintainability, test coverage and documentation
- Leading technical discussions to the right decisions (democracy of ideas)
- Reporting to the Head of Engineering
- At least 6 years' experience in architecting and monitoring complex yet scalable cloud data pipelines
- Expert in SQL and Python
- Strong understanding of design patterns and architectural best practices
- Experience of owning a product release process including design, development, QA, documentation, deployment and monitoring
- Ability to see the big picture and break it down into manageable chunks
- Experience of mentoring other data engineers
- Experience of real-time data processing (streaming)
- Ability to thrive in a fast-paced SCRUM environment
- Humility and desire to learn from anyone
- Infrastructure-as-code (AWS, Terraform)
Benefits: A very competitive salary awaits the right candidate. Remote working with periodic north London office visits (typically once a fortnight), a training budget for professional development, pension contributions etc.
Lead Data Engineer Data Pipeline Architect Big Data Engineer