Senior Data Engineer

About the job

Entity: Technology Job Family Group: IT&S Group Job Description:

Let me tell you about the role Our data engineers build and run the data pipelines and platforms that bp pulse relies on every day. The team builds on Databricks and AWS. As a Senior Data Engineer, you will design and build data pipelines, model data for analysis, and keep our platforms dependable and well-governed. It's a hands-on, senior role: setting a high engineering standard, mentoring other engineers, and acting as an informal technical lead within the squad leading through experience rather than a formal management line. What you will deliver

- Design, build and run reliable, scalable, distributed data pipelines on Databricks and AWS.

- Model data for analysis, keeping it accurate, well-governed and easy to use.

- Write secure, well-tested software to build, integrate and quality-assure data across bp pulse, meeting privacy and compliance needs.

- Follow and encourage good engineering practice: technical design and review, testing, code review, source control and documentation.

- Build and improve CI/CD pipelines and manage infrastructure as code.

- Look after the pipelines and services the team builds monitoring, alerting and business-hours support and keep improving their reliability.

- Mentor other engineers and share knowledge to lift the quality of the team's work.

- Work with people across the business to understand what they need and turn it into dependable data products. What you will need to be successful Essential

- Strong, hands-on experience building and running reliable, scalable, distributed data pipelines and data products in complex environments.

- Hands-on experience with Databricks, PySpark, Spark SQL, Delta Lake, Unity Catalog and Workflows (or similar lakehouse experience, with a real appetite to grow on Databricks).

- Experience with AWS data and serverless services (e.g. Lambda, Glue, S3, Redshift, Kinesis, SNS/SQS).

- Strong Python and advanced SQL.

- Good data modelling skills, applied in production.

- Experience with infrastructure as code and CI/CD.

- Experience mentoring engineers and helping raise the quality bar.

- Comfortable working with people across the business and shaping work through technical influence.

- A curious, continuous-improvement mindset.

- A degree in computer science or a related field, or equivalent knowledge and experience. Desired

- Experience with dbt or Apache Airflow.

- Familiarity with open table formats such as Delta, Apache Hudi or Apache Iceberg.

- Experience with Terraform or another infrastructure-as-code tool.

- No prior experience in the energy industry needed. What we offer In return, bp offers a competitive salary and bonus, a strong pension, private healthcare, flexible working and real support for your learning and career growth. [P&C to confirm country-specific benefits.] About bp Our purpose is to deliver energy to the world, today and tomorrow. For over 100 years, bp has focused on discovering, developing and producing oil and gas in the nations where we operate. We are one of the few companies globally that can provide governments and customers with an integrated energy offering. Delivering our strategy sustainably is fundamental to achieving our ambition to be a net zero company by 2050 or sooner! We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform crucial job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.

Travel Requirement No travel is expected with this role Relocation Assistance: This role is eligible for relocation within country Remote Type: This position is a hybrid of office/remote working Skills: Cloud Platforms, Cloud Platforms, Collaboration, Communication, Configuration management and release, Continuous deployment and release, Creating a high performing team, Database Design, Digital Project Management, Documentation and knowledge sharing, Emerging technology monitoring, Facilitation, Information Security, Mentoring, Metrics definition and instrumentation, NoSql data modelling, Problem Solving, Relational Data Modeling, Risk Management, Scripting, Secure development, Service operations and resiliency, Software Design and Development, Solution Architecture, Source control and code management {+ 5 more} . 
 Legal Disclaimer:

We are an equal opportunity employer. We do not discriminate on the basis of protected characteristics like race, religion, color, sex, national origin, sexual orientation, veteran status or disability status. Individuals with an accessibility need may request an adjustment/accommodation related to bp’s recruiting process (e.g., accessing the job application, completing required assessments, participating in telephone screenings or interviews, etc.). If you would like to request an adjustment/accommodation related to the recruitment process, please  contact us .

If you are selected for a position and depending upon your role, your employment may be contingent upon adherence to local policy. This may include pre-placement drug screening, medical review of physical fitness for the role, and background checks.