Uncubed
           

Big Data - Senior Engineer

Dathena, Singapore

Transform data from a liability to an asset


About Dathena Science
 
Dathena is a Swiss and Singaporean company developing data governance software based on machine learning algorithms. Dathena is the most complete and accurate data governance platform and is the only solution to easily classify and manage data with this level of accuracy while helping companies to comply with regulations. Using cutting edge machine learning technology, Dathena sift through all unstructured data allowing to regain full control on company’s information.

Job Purpose

  • We are looking for a talented Senior Big Data Engineer that will help us improve data ingestion and processing performance by analysing current Spark applications to design and implement efficient architecture solutions.
  • Your primary focus will be to build distributed Spark streaming pipelines with Kafka

Responsabilities

  • Design and implement streaming pipelines for a large application
  • Benchmark different solution approaches and analyze performances
  • Optimise code and resource usage
  • Present solutions approaches and architecture choices
  • Maintain high-performance and data integrity of critical database (NoSQL & SQL type)
  • Provide project technical support and expertise
  • Support new projects/integrations working with R&D team

Skills and qualifications

  • 5+ years experience in working on large projects in a Spark environment
  • Strong experience in Kafka integration with Spark Streaming
  • Experience in dealing with large HBase databases
  • Excellent functional programming skills in Scala
  • Knowledge of cloud computing infrastructure (e.g. Amazon Web Services EC2, Elastic MapReduce) and considerations for scalable, distributed systems.

Working conditions

  • Location: Singapore R&D Office

About Dathena

The company's founder created Dathena because he was not able to find an existing solution that would fit the needs of the top Tier 1 Bank he worked for. Dathena is the first and only data governance platform developed by real users for real users. Our platform was built with state-of-the-art technologies including machine learning and artificial intelligence capabilities.

Dathena

Want to learn more about Dathena? Visit Dathena's website.