Data Engineer II

About the job

Are you passionate about working with data at scale? Do you love bringing data together from diverse systems and sources and working on critical problems for understanding customer behaviour and generating actionable insights? Does the idea of partnering with a team of highly experienced machine-learning scientists and engineers excite you? Minerva Science and Analytics team is looking for an experience, self-driven Data Engineer to help us synthesise data into knowledge across a large number of businesses to help independent authors bring their creativity to customers, detect fraudulent and abusive behaviour, and democratise content creation in an safe, efficient, and exciting way. Our team has mature areas and green-field opportunities. We offer technical autonomy, value end-to-end ownership, and have a strong customer-focused culture.

Come join us as we revolutionise the book industry and deliver an amazing experience to our authors and readers.

Key job responsibilities As a Data Engineer on the Minerva Athena team, you will own and drive the design of scalable data systems keeping privacy and security compliance in check. You'll lead technical initiatives at the intersection of data engineering and AI, architecting solutions that process large-scale data. You'll mentor junior engineers, influence technical direction, lead design reviews and deliver high-impact data products with measurable business outcomes.

AI-Native Infrastructure 1. Design and architect AI-native infrastructure supporting real-time data processing for AI/ML inference, training, and continuous learning at scale. 2. Architect infrastructure for agentic AI systems with multi-agent orchestration, defining patterns and best practices for the team. 3. Drive GenAI-powered data quality, entity resolution, and metadata management strategies that raise the bar for data integrity

Data Governance 1. Own end-to-end accountability for data governance ensuring privacy/security compliance across datasets in cluster and data lake. 2. Design and build self-service platforms with embedded governance, lineage, and discovery — enabling teams to independently access and trust data 3. Define data contracts and API standards for reliable, versioned data consumption across downstream consumers

AWS Infrastructure & Pipeline Engineering 1. Architect and optimise AWS infrastructure: EC2, Lambda, S3, Redshift, EMR- balancing performance, reliability, and cost 2. Design high-throughput, fault-tolerant pipelines supporting BI Engineers, data/applied scientists, and AI agents at scale. 3. Drive infrastructure-as-code best practices using CDK, establishing reusable patterns and deployment standards

Technical Leadership & Operational Excellence 1. Mentor team members, conducting code reviews and elevating engineering standards 2. Own operational health of critical pipelines, driving root cause analysis and long-term prevention strategies 3. Author and maintain technical design documents, influencing architectural decisions across org. 4. Drive automation initiatives that reduce operational toil and enable non-linear scaling

About the team Minerva is a cross-functional team of highly experienced Applied scientists, Data Scientists, Data Engineers, BI Engineers and Software Engineers with a critical business mission making revolutionary leaps forward using massive-scale data with advanced analytics and machine learning and helping democratise the publishing industry. We build science-based systems for marketing and content-discovery for indie authors, fraud/abuse, and content risk. We also use science to optimise manufacturing, fulfilment, and quality processes for our Print-On-Demand business.

Basic qualifications - 3+ years of data engineering experience - 4+ years of SQL experience - Experience with data modeling, warehousing and building ETL pipelines

Preferred qualifications - Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions - Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)