As a remote, hourly paid Bash/PowerShell Engineer, you will review AI-generated responses and generate high-quality automation-focused content, evaluating the reasoning quality and step-by-step problem-solving behind scripts and troubleshooting workflows. You will assess solutions for accuracy, clarity, and adherence to the prompt; identify errors in logic, safety, or portability; fact-check technical details; write expert-level explanations and model solutions that demonstrate correct scripting patterns; and rate and compare multiple AI responses based on correctness and reasoning quality. This contract role is with SME Careers, a fast-growing AI Data Services company and subsidiary of SuperAnnotate that provides AI training data for many of the world’s largest AI companies and foundation model labs, directly helping to improve the world’s premier AI models.
Key Responsibilities
- Develop AI Training Content: Create detailed prompts in various topics and responses to guide AI learning, ensuring the models reflect a comprehensive understanding of diverse subjects.
- Optimize AI Performance: Evaluate and rank AI responses to enhance the model's accuracy, fluency, and contextual relevance.
- Ensure Model Integrity: Test AI models for potential inaccuracies or biases, validating their reliability across use cases.
Your Profile
- Expert-level proficiency in Bash and/or PowerShell scripting.
- 2–3+ years of hands-on experience in at least one of Bash or PowerShell.
- Strong experience writing, debugging, and maintaining shell or object-based automation scripts.
- Ability to evaluate scripts for correctness, readability, safety, performance, and portability across environments.
- Professional experience in software engineering, DevOps, IT automation, or infrastructure-focused roles.
- Minimum Bachelor’s degree in Computer Science or a closely related technical field.
- Significant experience using large language models (LLMs) while coding, troubleshooting, and reviewing scripts.
- Excellent English writing skills with the ability to document and explain complex automation clearly.
- Highly preferred: For Bash, strong comfort with awk, sed, and grep; for PowerShell, experience with PowerShell Core, DSC, and Azure PowerShell modules.
- Preferred: Merged PRs in CLI, DevOps, or infrastructure-related open-source projects; familiarity with CI pipelines, automation workflows, or cloud scripting; previous AI data training or model evaluation work strongly preferred and Minimum C1 English proficiency.
