Location
London

Hours
Full Time

Salary
Negotiable

About the Role
The Climate Data Utility Tech Team is seeking an experienced Senior Data Engineer to play a pivotal role in designing, implementing, enhancing, and maintaining scalable data pipelines for the Net-Zero Data Public Utility. These pipelines support the Utility's mission to provide open and accessible public good data via the Climate Data Utility website and APIs. You will work with diverse data sources, each with unique formats, fields, and access protocols, managing the full data lifecycle from extraction, transformation according to source and domain-specific business logic, to ingestion. Your responsibilities include implementing data quality checks and validation procedures to ensure the accuracy and reliability of the data provided by the Utility.

Key responsibilities include designing and developing scalable data pipelines using Python to read from APIs and structured files such as Excel and Parquet, translating domain and source-specific business logic into efficient code, and applying data transformations and structuring with tools like Pandas and Pydantic. You will collaborate closely with data scientists and analysts to support data-driven decision-making, document data engineering processes including data lineage, data dictionaries, and system architectures, and maintain best practices through code reviews, version control, and adherence to industry standards. Additionally, you will deploy production-quality code through CI/CD pipelines into our cloud environment and provide mentorship and guidance to junior team members to foster their growth in data engineering practices.

Experience
7+ years of experience in data engineering or a similar role. Proven proficiency in Python programming and experience building and managing data pipelines. Knowledge of data warehousing and ETL processes is essential. Strong problem-solving and communication skills are required. Note that years of experience are a guide; applications from candidates demonstrating the necessary skills will be considered.

About you
You are a motivated and detail-oriented professional with a passion for data engineering and climate-related data initiatives. You thrive in collaborative environments, enjoy mentoring others, and are committed to delivering high-quality, reliable data solutions that support public good.

Qualifications
Relevant qualifications or equivalent experience in data engineering, computer science, or related fields are preferred but not mandatory. A strong portfolio of data engineering projects and demonstrated expertise in Python and data pipeline development is highly valued.