- Work directly with Machine Learning Engineers and Platform Engineering Team to create reusable experimental and production data pipelines.
- Understand, tune, and master the processing engines (like Spark, Hive, Samza, etc) used day-to-day.
- Keep the data whole, safe, and flowing with expertise on high volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc).
- Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit.
- Explore as many new technology options for data processing, storage, and share them with the team.
- Develop tools and contribute to open source wherever possible.
- Adopt problem solving as a way of life – always go to root cause
- Degree in Computer Science, Engineering or a related field
- You have previously worked on building serious data pipelines ingesting and transforming > 10 ^6 events per minute and terabytes of data per day.
- You are passionate about producing clean, maintainable and testable code part of real-time data pipeline.
- You understand how microservices work and are familiar with concepts of data modelling.
- You can connect different services and processes together even if you have not worked with them before and follow the flow of data through various pipelines to debug data issues.
- You have worked with Spark and Kafka before and have experimented or heard about Flink/Druid/Ignite/Presto/Athena and understand when to use one over the other.
- On a bad day maintaining zookeeper and bringing up cluster doesn’t bother you.
- You may not be a networking expert but you understand issues with ingesting data from applications in multiple data centres across geographies, on-premise and cloud and will find a way to solve them.
- Proficient in Java/Scala/Python/Spark
What we Offer!
- Due to the pandemic, we have been and will continue to WFH until it is safe to open our office. Our company culture and values remain at the core of everything we do.
- For the third year in a row, we are proud to announce that we have been certified as a Great Place to Work
- We were also certified as one of the Best Workplaces for Mental Wellness in 2020
- We are an open work environment that fosters collaboration, ownership, creativity, and urgency
- We ensure flexible hours outside of our core working hours
- Enrolment in the Group Health Benefits plan right from day 1, no waiting period
- To keep things fun and stress-free during COVID-19 we started Virtual Daily, Virtual Weekly and Monthly team bonding activities including: Trivia, Games Nights, Movies Nights, Arts & Crafts (e.g. Origami), Lunch & Learns (e.g. Sign Language 101), Virtual Wellness Sessions (e.g. Meditation, Morning stretches), Virtual Team Ubereats Lunches, and so much more
- We also created and began publishing a monthly internal newsletter with various topics that keeps the tone lighthearted and interesting
- Team building events (anything from axe throwing, go-karting, bike riding, etc.)
- Fuel for the day: Weekly delivery of groceries, and all types of snacks
- Catered lunches and desserts on a monthly basis
- Flexibility with WFH
- Daily fun in the office with our competitive games of Ping Pong, Pool, Smash Bros competitions, or FIFA
- And of course, an unlimited amount of freshly made coffee! We’re pretty serious about our coffee beans
About Paytm Labs
About us Paytm Labs builds technologies that power Paytm, the world's fastest growing mobile payments and commerce ecosystem. We use our skills, and our biggest asset - data, to make our little dent in this universe. We make commerce smoother, safer, and personal. Learn more about us at http://www.paytmlabs.com/.
Want to learn more about Paytm Labs? Visit Paytm Labs's website.
Reddit is an American social news aggregation, web content rating, and discussion website.