SparkPost, a Message Bird company, is the industry’s most trusted email optimization platform. SparkPost helps senders reliably reach the inbox with powerful solutions to help them plan, execute and optimize their email programs. The SparkPost platform is powered by the industry’s largest data network, a team of email experts to help brands elevate every aspect of their email program, and a security and compliance posture to support even the most regulated industries. SparkPost is the world’s largest sender, delivering 40% of all commercial email - 4-5 trillion sends annually - and also boasts the world’s largest data footprint to help enterprise-level brands make data-driven decisions to improve their email performance. The world’s most sophisticated senders, including The New York Times, Zillow, Adobe and Booking.com trust SparkPost to elevate their email. 

We provide cloud-based data analytics and insights to our customers for better email inboxing, engagement, and conversion. As part of that mission, we collect and analyze vast quantities of event data to help us understand a customer’s current performance and how to drive improved performance and ROI.

We're looking for a Lead Data Engineer to join our SparkPost engineering team working remotely within the United States.

Who You Are

• You have a passion for building innovative customer-facing data products, including launching and supporting in production.
• You love working on challenging problems involving scalability and performance.
• You have several years of practical production experience at scale with various types of data pipelines & ETLs for batch and real-time data processing.
• You have production experience running Apache Spark as well as AWS S3 data lakes, EMR, and Glue
• You are experienced in designing & loading data into various types of data marts & data warehouses (relational & non-relational), and writing & tuning queries.
• You are very competent with Linux and have worked in an AWS environment.
• You are familiar with CI/CD for both software and database schemas and ideally experienced with containers, serverless, and microservices.
• You enjoy learning new technologies and picking up new skills and love to mentor and teach others.

What The Role Is

• Design, build, deploy, and support scalable data pipelines and data storage for structured and unstructured data queries & analysis in support of rapidly delivering high-impact customer-facing data products.
• Collaborate within a small agile team and with other teams & functions (e.g. Data Science & Product Management, UX Design, and Front-end engineering) to deliver high-impact results.
• Leverage DevOps techniques and practices like Continuous Integration, Continuous Deployment, Test Automation, Build Automation and automated unit/functional testing, microservices, containers, and serverless to rapidly deliver working code to production.  
• Use sound judgment and prototyping to make build vs buy tooling decisions, with consideration of time to market.
• Conduct architecture, design, and code reviews with other team members to make sure code is rigorously designed, elegantly coded, and effectively tuned for performance

Qualifications

• Bachelor’s Degree in Computer Science or related technical discipline.
• At least 6 years of data pipeline & data infrastructure experience in production
• At least 3 years of experience working with some of the following: Apache Spark (required), Flink, Storm, TensorFlow, Kafka, Presto, AWS Athena, AWS Glue, and AWS Redshift, AWS Lambda
• Expertise with RDBMS, SQL, and NoSQL including columnar data stores
• At least 8 years of programming experience, with Python and Node.js preferred.
• At least 3 years of experience working with Linux-based OSes
• At least 5 years of experience working within cloud environments, preferably AWS