Lead Data Engineer

Role: Lead Data Engineer Location: London (Hybrid - 1 day per week in office) Are you ready to design, implement, and manage secure, scalable data solutions in off-cloud environments? We're looking for an experienced Lead Data Engineer to join our team and take the lead in optimising our data infrastructure. With a strong focus on PostgreSQL, PostGIS, and TimescaleDB, you will design APIs, manage data orchestration with AirFlow and SQLMesh, and prepare our systems for key security accreditations. This role is ideal for someone passionate about data engineering and excited to work alongside technical leads to support our Data Sciences Roadmap. What You'll Do: Database Design & Optimisation : Architect and maintain highly efficient databases using PostgreSQL, PostGIS, and TimescaleDB, focusing on scalability for geospatial and time-series applications. API Development : Design and build secure, scalable APIs to streamline data flow between frontend and backend systems. Data Orchestration : Manage and optimise data pipelines using AirFlow and SQLMesh for smooth workflow automation. Off-Cloud Deployment : Use Kubernetes (Talos, RKE2) and storage solutions like Ceph for deploying applications and databases in off-cloud settings. Schema Management : Use Prisma for effective schema management, ensuring database efficiency and version control. Security & Compliance : Drive system architecture towards accreditation standards by implementing best practices in data security, encryption, and role-based access control (RBAC). Collaborate with the Principal Engineer : Work closely with our Principal Engineer on monitoring, DevSecOps, and system performance to ensure optimal integration and functionality. What You Bring: Extensive Expertise in PostgreSQL/PostGIS, TimescaleDB, and API design. Off-Cloud Architecture Skills : Experience with Kubernetes (Talos, RKE2) and storage solutions like Ceph. Data Orchestration : Proficiency with AirFlow and SQLMesh. Strong Security & Compliance Knowledge : Experience with data encryption, RBAC, and secure systems design. Performance Optimisation : Expertise in query optimisation, backup strategies, and clustering for high availability. Monitoring & ETL : Experience with tools like Prometheus, Grafana, and database migration/ETL pipelines. Bonus Skills: Familiarity with NoSQL databases (e.g., MongoDB, Cassandra) and schema management tools like Prisma, Liquibase, or Flyway. A knack for collaborating with DevOps teams to ensure system reliability and disaster recovery. What We Offer: Hybrid Work Model : Balance your time between our London hub and the comfort of your home. Generous Leave : 25 days of leave + bank holidays (and your birthday) Other Benefits : Including health insurance, access to a mental health support app, and equity options. Exciting Challenges : Be a part of a team that’s preparing for future security certifications and authorisations for public and private sector organisations. Important: Applicants must be UK residents with the right to work in the UK, as SC clearance may be required for this role in the near future. Ready to make an impact as our Lead Data Engineer? Apply now and join us on this exciting journey!