Principal Data Engineer
White Ops, US Remote or Canada Remote
We love the internet and protect it accordingly.
At White Ops, we are all about keeping it human. We are the global leader in bot mitigation. We protect more than 200 enterprises—including the largest internet platforms—from sophisticated bots by verifying the humanity of nearly one trillion online interactions every week. The most sophisticated bots look and act like humans when they click on ads, visit websites, fill out forms, take over accounts, and commit payment fraud. We stop them.
Founded in 2012 in a Brooklyn sci-fi bookstore, our Bot Mitigation Platform protects enterprises from the sophisticated bots that threaten them. It’s an ongoing war that we fight passionately every day. Join our mission to stop bots, disrupt the economics of cyber crime, and keep it human.
The Data Engineering team is responsible for the primary organs and arteries of our system. The team’s mission is to consistently deliver a world-class data platform that enables our security researchers, threat intel analysts, and data scientists to quickly and reliably detect malicious non-human activity. You will be solving a wide range of problems from finding more efficient ways to research sophisticated botnets to delivering only the most critical and impactful insights to our customers as fast as possible so they can take action.
By joining the Data Platform team at White Ops, you will gain a deep understanding of how our products and services work all the way from signal collection to customer reports and are able to make a major impact on cleaning up the internet. You will be responsible for the evolution of a platform that will grow in scale by 100x (current volume is over 1 PB) to 1000x, ensure that our data can never be used for ill by anyone (e.g. privacy and security are at the core of the system design), and incorporate the latest ML and fraud research techniques.
What you'll do:
- Architect, Design, Build, Test, Deploy and Maintain reliable, scalable systems and applications to process our current and expected future loads
- Empower our stakeholders to automate and enhance their day to day data requirements (finance, marketing, sales, product management, etc.) leveraging frameworks and libraries
- Partner closely with security researchers, threat intel analysts, and data scientists to understand how they work and uncover new ways to enhance and amplify their analysis
- Lead and manage the ingestion and usability of a Snowflake data warehouse that ingests and process very high volume near real time data
- Partner with our operations teams to debug and resolve critical issues and participate in an on-call support rotation for mission critical production applications
- Work with a team of uniquely talented engineers who love to learn, experiment, and push the limits of modern technology
- Leverage DevOps Practices and Principles to build and leverage CI/CD pipelines
Who you are:
- You have considerable experience in designing and architecting highly scalable and resilient big data eco systems leveraging large data sets (order of 100 billion interactions and multiple petabytes) successfully and hands on experience developing them
- You have demonstrated the ability to design performant processes that support near real time data transformation, data structures, metadata, data quality frameworks, dependency and workflow management leveraging relational, NoSQL and other big data technologies (SnowFlake experience is preferred)
- You have expertise in developing software in Java, Python, Java Framework and Spring understanding both the theoretical concepts of the language as well as how to write well-documented well-tested code
- You excel in automating the development and deployment of distributed software to cloud environments like AWS, Azure or GCP (which likely also means you know some combination of Kubernetes, Docker, and/or Terraform)
- You have experience in efficiently analyzing and extracting key insights from large (terabytes/petabytes) datasets
- You have the ability to research, propose, develop and deploy new technologies and have ability to clearly articulate pros and cons of various technologies and platforms and to document use cases, solutions and recommendations
- You can coach and mentor other team members and are key in setting up standards, principles and best practices in design, architecture, coding, testing and deployment
- You possess skills and exhibit behaviors that cultivate and drive us towards technology modernization and strong NFRs involving scalable design and integration patterns
Benefits and Perks
- Unlimited vacation policy
- Stock options, 401(k), and commuter benefits
- Competitive salary and benefits
- Medical and dental insurance for all full-time employees
- Fully paid parental leave
- Professional development fund
- Great coaching from senior leaders and challenging development opportunities
Life at White Ops:
Our HQ office is located in the heart of New York City. We are growing the company deliberately with a keen eye towards maintaining a culture that values diversity, lifestyle, and career growth. We are doing meaningful work and we need people to join our mighty team. We are proud of our overwhelmingly positive presence on Glassdoor and Built in NYC. We have offices located in NYC, DC, Victoria, and London.
About White Ops
ABOUT WHITE OPS We are a cybersecurity company that protects digital advertisers and web app owners from automated threats: threats like ad fraud, account stuffing, or fake engagement. Our products accurately detect and prevent bots even in the presence of sophisticated cybercriminal techniques like exploiting real people’s devices or simulating human behavior. Our customers include some of the largest and most forward-thinking companies on the web.
Want to learn more about White Ops? Visit White Ops's website.
A membership platform that makes it easy for artists and creators to get paid