Big Data Platform Architect (Principal Engineer)
A³ by Airbus Group, Sunnyvale, CA
The Future of Flight
Project Wayfinder is building scalable, certifiable autonomy systems to power the next generation of commercial aircraft. Our team of experts is driving the maturation of machine learning and other core technologies for autonomous flight; we are creating a reference architecture that includes hardware, software, and a data-driven development process to allow aircraft to perceive and react to their environment. Autonomous flight is transforming the transportation industry, and our team is at the heart of this revolution.
As a Big Data Platform Architect, you will be responsible for the design, implementation and continued optimization of a scalable data storage and compute infrastructure at Airbus and Acubed data centers to support Wayfinder's data-driven development process for vision-based autonomy functions. This entails:
- Aggregating and storing tens of petabytes of data from a wide range of sources
- Enabling worldwide data collection and aggregation at scale
- Making data available to various groups at Airbus
- Tailor the solution to scale and automate a data-driven development process to fuel vision-based machine learning algorithms (e.g., autonomous taxi, takeoff and landing) for autonomy functions.
You will be responsible for designing, prototyping and demonstrating the feasibility and scalability of a hybrid cloud and on-premise infrastructure by implementing and running a pilot version at Acubed. The pilot version will provide guidance and expertise to the global Airbus organization in Europe and serve the operational needs of the Wayfinder team.
You will also be in charge of working with various Airbus groups in the US and Europe to evaluate the applicability of existing solutions and technologies, and to plan and execute on the expansion of the infrastructure solution.
- Lead the design, prototype and expansion of a scalable data infrastructure to support the development and testing of autonomous flight functions (e.g. automated taxi, takeoff and landing) across different data centers and Airbus groups in the US and Europe
- Design and prototype a data infrastructure to aggregate and store tens of petabytes of data (e.g. images, videos and point clouds) as well as train and test machine learning algorithms at scale
- Perform cloud vs. on-premise infrastructure trade-offs for an optimal and cost-effective solution that will support software- and hardware-in-the-loop testing
- Interface with vendors and different stakeholders at Airbus to establish a global solution.
- Bachelor’s degree in computer science, computer engineering or a related discipline
- 8+ years of professional experience in large scale data infrastructure (i.e. experience in designing on-premise and cloud-based Infrastructure-as-a-service or platform-as-a-service solutions)
- 2+ years managing small to large teams in a dynamic environment
- Demonstrated ability to design, build, maintain and improve large scale infrastructure (e.g. ActiveScale, WekaIO, SLURM, Kubernetes, AWS, GCP)
- Prior experience and expertise in distributed data storage (e.g. S3, HDFS, NFS)
- Prior experience and expertise in GPU clusters (on-premise and cloud-based)
- Documented proof of fully vaccinated status required (or qualify for an exemption)
Strongly preferred qualifications
- Experience with infrastructure for machine learning based autonomous vehicle applications, including GPU accelerated computing
- Real-world experience with infrastructure supporting multi-petabyte datasets
- Experience with tools for storing and managing large scale data (e.g., Apache Parquet, Postgresql, ELK, Spark)
- Exceptional PPO medical, dental and vision benefits 100% of premiums covered for employee and their family/dependents
- Generous PTO of 5 weeks (6 weeks after two years) in addition to 11 national holidays and unlimited paid sick days
- Tuition reimbursement for professional development or $15,750 for flight training
- 3 months paid parental leave from Day 1
- Employee discounts through Airbus Tickets @Work, gym subsidy/membership and more
About A³ by Airbus Group
Mission Imagine a world without boundaries, free of constraints. It’s a place to innovate, to discover, to create, to explore. Enter this world with your eyes wide open, with your ideas intact as you journey into the unknown. Dare to redefine the future. We are the Silicon Valley outpost of Airbus, a global leader in aviation and aerospace innovation. We believe that the future is created through episodic disruption with intervening periods of incremental innovation. Our mission is to build the future of flight now. We solve big problems by employing world-class experts and by developing fertile partnerships. Our projects are built on rigorous analysis, novel insights, and commitment to unreasonable goals. Exploration is unsettling, and only when we disrupt Airbus, its competitors, and the entire industry will we consider our goals achieved and our mission a success. Join us on our bold voyage.
Want to learn more about A³ by Airbus Group? Visit A³ by Airbus Group's website.
Lyft is your friend with a car, whenever you need one