Senior Big Data - Engineer
Transform data from a liability to an asset
- We are looking for a talented Data Engineer that will help us improve data ingestion and processing performance by analyzing current Spark applications to design and implement efficient architecture solutions.
- Design and implement data pipelines for large applications
- Responsible for systems analysis - Design, Coding, Unit Testing and other SDLC activities
- Benchmark different solution approaches and analyze performances
- Optimise code and resource usage
- Present solutions approaches and architecture choices and make sure data architecture standards are adhered
- Maintain high-performance and data integrity of critical database (NoSQL & SQL type)
- Implement and update ETL processes
- Provide project technical support and expertise
- Support new projects/integrations working with R&D team
Skills and qualifications
- Master’s Degree in Computer Science
- 5+ years experience in working with big data ecosystems in a Spark and Hadoop environment
- Start-up experience is a huge plus
- Understanding of databases fundamentals, distributed computing and micro service architecture principals. Experience in dealing with large HBase databases
- Functional programming skills in Scala and/or Java is good to have
- Experience with Docker, Unix Shell (Bash), Python. Kubernetes is a plus.
- Good understanding of Data Science concepts (e.g. Machine Learning and Natural Language Processing)
- Knowledge of cloud computing infrastructure (e.g. Amazon Web Services EC2, Elastic MapReduce) and considerations for scalable, distributed systems are a plus
- Requirement gathering and understanding, Analyze and convert functional requirements into concrete technical tasks and able to provide reasonable effort estimates
- Work proactively, independently and with global teams to address project requirements, and articulate issues/challenges with enough lead time to address project delivery risks
- Providing expertise in technical analysis and solving technical issues during project delivery
- Code reviews, test case reviews and ensure code developed meets the requirements
- Experience working in Agile/Scrum teams using JIRA, Git, Bitbucket
- Location: Singapore R&D Office
The company's founder created Dathena because he was not able to find an existing solution that would fit the needs of the top Tier 1 Bank he worked for. Dathena is the first and only data governance platform developed by real users for real users. Our platform was built with state-of-the-art technologies including machine learning and artificial intelligence capabilities.
Want to learn more about Dathena? Visit Dathena's website.
File hosting service