Data Engineer

CB Insights, New York

Software that predicts technology trends.​

Want to build a product that uses data to see and make sense of the future?

If you are a coding fanatic and passionate about programming, we want you to help us make a huge impact. Our clients love our product and are thirsty for more!

At CB Insights we build products that help clients make sense of the future and drive their businesses forward using data. Our system retrieves large amounts of structured and unstructured data and uses scientific methods to extract knowledge and insights from that data. We present those analytics through a sophisticated, dynamic user interface which enables our clients to find answers to their most important questions.

As a Data Engineer, you will be a core part of a strong team building data pipelines and a robust infrastructure enabling effective data processing at scale. You will build products that use natural language processing and machine learning models and make them run efficiently with large amounts of data to enable a smooth experience for our clients and in-house intelligence units. We focus on modularity and reuse where it makes sense while ensuring that there are no constraints to delivering world-class software continuously.

We’re looking for engineers that through hard-won practical experience know how to build maintainable and testable data pipeline processes and infrastructure. We are looking for engineers that love solving problems and are willing to take on hard ones.

Much of our software team has been with us for several years, despite a white hot tech market with options galore. We attribute this to our collaborative teach and learn culture where the role evolves with your interests.

If this sounds interesting to you, reach out and join CB Insights now!

Key Responsibilities:

  • Engineer efficient, adaptable and scalable data pipelines that power our data products
  • Design and build efficient ETL infrastructures for unstructured textual data sets and various other types of data sources
  • Take a prototype of a data product built with NLP and/or machine learning models and make it run reliably in production.
  • Monitor and maintain existing data products running in production including identifying when models need to be retrained
  • Design and implement internal tools to make this data processing infrastructure easily accessible to and usable by other software developers
  • Develop solutions that are well-engineered, maintainable, tested and delivered on time.
  • Participate in code reviews and sprint planning, help to identify problems and share knowledge with your colleagues.

Required Experience & Qualifications:

  • 2+ years professional software/data engineering experience using Python, SQL and at least 1 statically typed language (Go, Java, Scala)
  • Knowledgeable about data modeling, data storage techniques, data warehousing and general data architecture
  • Experience with engineering data pipelines to capture, store and process unstructured data
  • Excellent written and verbal communication skills
  • Excellent problem solving and analytical skills
  • Believer in Lean and Agile values and principles for building software
  • Proficiency developing in a Mac/Linux environment
  • Technologies/Languages: Python, Go, SQL, NoSQL, Spark, Hadoop
  • Helpful Humble Human

Nice to Have/s:

  • Experience with Go, AWS services (RDS, S3, SQS, Redshift, Spectrum, Glue)
  • Experience building and maintaining a Hadoop or Spark cluster and other related tools in the big data ecosystem

Perks and Benefits:

  • Subsidized health, dental and vision insurance
  • 401K with up to 4% match
  • $1,000 yearly continuing education stipend
  • Daily lunch stipend

Equal Opportunity Employer:  CB Insights is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status.


About CB Insights

CB Insights has built a tech market intelligence platform that analyzes millions of data points on venture capital, startups, patents, partnerships and news media to predict technology trends.

Be a Better CB Insights Candidate

Learn skills and get an insider's look at CB Insights when you watch classes taught by their top employees.

Want to learn more about CB Insights? Visit CB Insights's website.