Job Title: Software Engineer
Location: Remote (Any time zone /Part time, 20 hrs/week)
Duration: 08 Months 
 
Duties:
Create a framework for managing jobs on a combination of on-premises and external cloud compute, where a job is allocated a compute node to load an LLM, retrieve queries from remote storage, and generate responses from the LLM given the queries. The framework must be robust and include fault tolerance so that jobs can be restarted/recovered upon failure and child nodes are brought down cleanly upon success.
 
Required skills include:

• Experience working with Python
• Experience working with LLM inference libraries (e.g. vLLM, transformers, or nemotron)
• Experience with Kubernetes, including managing tasks given available infrastructure, and coordinating containers
• Experience building robust distributed applications that allow graceful failure and recovery
• Experience working with AWS/GCP cloud platforms
• Experience using basic git functionality, including branching, pull requests, reviews, and merging
• Experience writing unit tests
• Excellent collaboration skills are required
• 2-3 years of working experience

Bonus skills include:

• Experience with PyTorch
• Experience with API design

 
 
At SPECTRAFORCE, we are committed to maintaining a workplace that ensures fair compensation and wage transparency in adherence with all applicable state and local laws. This position’s starting pay is: $ 50.00/hr.