This is an exciting opportunity to solve challenging problems and make a big impact as we invest in making technographics a first-class offering.

Key Responsibilities:

• Build and optimize big data pipelines to extract and process signals from the web, job postings, and other sources
• Design and implement data architectures and storage solutions to efficiently handle massive data volumes
• Collaborate closely with data scientists to support and integrate ML models into data workflows
• Continuously improve data quality, performance, and scalability of our technographic data platform
• Drive technical strategy and roadmap for the data processing infrastructure

Requirements:

• Extensive experience building and scaling big data pipelines and architectures from scratch
• Deep expertise in big data frameworks (Hadoop, Spark) and the JVM stack (Java, Scala)
• Strong software engineering fundamentals and ability to write efficient, high-quality code
• Experience with entity recognition and NLP techniques a plus
• Proven track record delivering results and driving projects in a fast-paced environment
• Excellent collaboration and communication skills to work with data scientists, analysts and product teams
• Passion for leveraging huge datasets to power valuable insights

Ideal Background:

• 8+ years of experience in software engineering roles
• Experience working with very large datasets and distributed systems
• Familiarity building data pipelines at large tech companies or data-driven organisations
• Bachelor's or advanced degree in Computer Science, Engineering or related technical field

XML job scraping automation by YubHub