Senior Infrastructure Engineer – GCP / Dataproc

6-month contract | Remote in NA | C2C

We’re working with a major media and entertainment company looking for a Senior Infrastructure Engineer to join their Infrastructure team on a 6-month contract.

The primary focus of the role is a Dataproc 2.3 to 3.0 upgrade, alongside remediation of infrastructure security findings across their GCP environment.

Responsibilities

• Lead the upgrade of Google Cloud Dataproc from version 2.3 to 3.0.
• Assess Dataproc version compatibility and define the infrastructure changes required to support the upgrade.
• Manage Dataproc cluster configuration, including initialization scripts and platform configuration.
• Test upgrade paths and validate workloads across Dataproc versions.
• Develop and maintain Dataproc plugins where required.
• Identify and resolve compatibility issues arising from Dataproc version changes.
• Investigate and remediate infrastructure security findings, prioritising risks and coordinating fixes.
• Implement infrastructure changes using Terraform and/or AWS CDK.
• Coordinate testing and validation with adjacent engineering teams and partners.
• Plan and manage rollout windows across multiple teams.
• Ensure infrastructure changes are documented, auditable and reproducible.

Required experience

• Strong hands-on experience with GCP and Dataproc in a production environment.
• Experience managing Dataproc versions, cluster configuration and upgrade paths.
• Experience with Dataproc initialization scripts and/or plugin development.
• Strong understanding of compatibility testing across platform versions.
• Terraform and/or infrastructure-as-code experience.
• Experience identifying, triaging and remediating infrastructure security findings.
• Experience coordinating infrastructure changes across multiple engineering teams.
• Comfortable independently owning infrastructure work from assessment through testing and production rollout.