Job Title: Data Engineer
Location: Remote
Duration: 8 months
 
Job Description

• Design, develop, and optimize Spark applications using Scala and Python.
• Build scalable and high performance ETL/ELT pipelines for batch and streaming data.
• Work with large datasets in distributed environments.
• Develop and deploy data pipelines using AWS services such as:
• S3, EMR, Glue, Lambda, Athena, Redshift, Kinesis
• Implement CI/CD pipelines for data workflows using AWS-native tools or DevOps frameworks.
• Manage infrastructure-as-code using Terraform or CloudFormation (preferred).
• Work with structured, semi structured, and unstructured data.
• Implement data quality checks, validation frameworks, and metadata management.
• Optimize Spark jobs for performance, memory usage, and cost efficiency.
• Partner with data scientists, analysts, and business teams to deliver data solutions.
• Troubleshoot production issues and ensure high availability of data pipelines.
• Document technical designs, workflows, and best practices.       

 
 
At SPECTRAFORCE, we are committed to maintaining a workplace that ensures fair compensation and wage transparency in adherence with all applicable state and local laws. This position’s starting pay is: $ 50.00/daily.