Job Title: Software Engineer III (Synthetic Data Infra)
Duration: 12 Months
Location: Remote
Summary:

• We are seeking a skilled and experienced engineer to join our team as a Distributed Systems Specialist. As a generalist with expertise in distributed systems, you will play a key role in designing, developing, and optimizing our data generation pipelines for maximum efficiency. Your experience with DAGs/Graphs, large fan-out loads, and distributed workers/jobs will be instrumental in helping us scale our systems to meet the growing demands of our users.

Must-Have Skills:

• Experience running compute clusters or building data infrastructure
• Proficiency in at least one of the following programming languages: Python (ideally), Rust, C++, Hack (PHP)
• Experience with cloud-based infrastructure such as AWS or GC

Nice-to-have Skills:

• Experience with companies using large data scales
• Experience with Digital Content Creation (DCC) tools such as blender
• Experience with machine learning frameworks, such as TensorFlow or PyTorch

Years of overall experience required:

• 5+ years

Degrees/certifications required:

• Bachelor’s with tech field related

Responsibilities:

• Design, develop, and optimize data processing pipelines using DAGs/Graphs and distributed workers/jobs
• Collaborate with cross-functional teams to identify and prioritize system improvements
• Develop and implement solutions to handle large fan-out loads and ensure system scalability
• Work closely with data scientists and engineers to integrate and scale solutions
• Identify opportunities for optimization and implement changes to improve system performance and reduce resource utilization.

Requirements:

• 5+ years of experience in software development, with a focus on distributed systems
• Experience with large fan-out loads and distributed workers/jobs
• Proficiency in at least one of the following programming languages:

• Python (primary language)
• Rust
• C++
• Hack (PHP)

• Experience with cloud-based infrastructure, such as AWS or GCP
• Strong problem-solving skills and ability to work independently
• Excellent communication and collaboration skills.

Surrounding team & key projects:

• Focused on rendering infrastructure, the team generates large volumes of computer-generated data to support the machine learning pipeline. The necessary infrastructure for accessing compute resources is in place, enabling a seamless user experience in real-life (RL) applications.

Typical Day-to-Day in the role:

• SW developments in scheduling system
• Authorize and submit data pipelines
• Work “On call”
• Collaborate with engineers who are writing pipelines

How will performance be measured:

• Ability to work independently and track down problems and improve systems
• Strong proven technical ability

What makes this role interesting:

• This position offers the opportunity to work on systems that handle large-scale compute workloads, providing significant challenges in scalability and performance. You will take ownership and be responsible for maintaining and optimizing code written by others, fostering collaboration and shared success. The role requires a proactive mindset to seek efficiency improvements across a wide range of workloads, making a meaningful impact on system reliability and resource utilization.

Interview Process:

• How many rounds of interviews: 2
• Types of Interviews 1 coding, 1 behavioral
• Interview Duration 45 min for coding, 30 min behavioral

 

 


At SPECTRAFORCE, we are committed to maintaining a workplace that ensures fair compensation and wage transparency in adherence with all applicable state and local laws. This position’s starting pay is: $70.00/hr.