Data Engineer

CB Insights, New York

Software that predicts technology trends.​

Build data-driven products and help us predict the next big thing.

At CB Insights, we build products to gauge and predict technology trends. This requires gathering information from disparate sources, analyzing it, extracting useful information and surfacing that on our platform. As a data engineer at CBI, you will be a core part of this process end-to-end and help us in building data pipelines and the infrastructure that enables this. You will help build products that use natural language processing and machine learning models and make them run efficiently with large amounts of data to enable the best user experience whether they be end-users or our data analysts.

We’re looking for engineers that, through hard-won practical experience, know how to build maintainable and testable data pipeline processes and infrastructure. We are looking for engineers that love solving problems and are willing to take on hard ones. Sounds a tad cliché but as engineers, we believe that the best professional satisfaction comes from knowing our customers use the software we’ve built and love it.

Key Responsibilities:

  • Engineer efficient, adaptable and scalable data pipelines that power our data products 
  • Design and build efficient ETL infrastructures for unstructured textual data sets and various other types of data sources 
  • Take a prototype of a data product built with NLP and/or machine learning models and make it run reliably in production.
  • Monitor and maintain existing data products running in production including identifying when models need to be retrained
  • Design and implement internal tools to make this data processing infrastructure easily accessible to and usable by other software developers
  • Develop solutions that are well-engineered, maintainable, tested and delivered on time.
  • Participate in code reviews and sprint planning, help to identify problems and share knowledge with your colleagues.

Required Experience and Qualifications:

  • 2+ years software/data engineering experience
  • 2+ years professional experience with using Python, SQL
  • Knowledgeable about data modeling, data storage techniques, data warehousing and general data architecture
  • Experience with engineering data pipelines to capture, store and process unstructured data
  • Experience with building and maintaining a Hadoop or Spark cluster and other related tools in the big data ecosystem
  • Excellent written and verbal communication skills
  • Excellent problem solving and analytical skills
  • Proficiency developing in a Mac/Linux environment
  • Technologies/Languages: Python, SQL, NoSQL, Spark, Hadoop
  • 4H's:  Happy, Helpful, Humble and Hungry

CB Insights values diversity, different perspectives, collaboration, and curiosity.

Additional Qualifications:

  • Experience with Go, Scala

Perks and Benefits:

  • Subsidized health, dental and vision insurance
  • 401K with up to 4% match
  • $1,000 yearly continuing education stipend
  • Daily lunch stipend

Equal Opportunity Employer:  CB Insights is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.

If you know someone who'd be perfect for the role, 
submit here and you'll be eligible for $5,000!

About CB Insights

CB Insights has built a tech market intelligence platform that analyzes millions of data points on venture capital, startups, patents, partnerships and news media to predict technology trends.

Be a Better CB Insights Candidate

Learn skills and get an insider's look at CB Insights when you watch classes taught by their top employees.

Want to learn more about CB Insights? Visit CB Insights's website.