Job Description

Role Overview

Create rigorous, real-world social science research evaluations that help train and benchmark next-generation AI systems. You will translate domain expertise into authentic research tasks, supporting materials, and detailed scoring frameworks. Prior AI experience is not required.

Key Responsibilities

• Design and author complex evaluation tasks based on real research scenarios, including survey analysis, qualitative coding, literature reviews, and statistical analysis.
• Source, synthesize, and create realistic research artifacts such as survey instruments, coded datasets, statistical outputs, and source documents.
• Define suitable methodologies and provide clear, defensible interpretations aligned with social science best practices.
• Create comprehensive grading rubrics with 35 or more items that assess methodological choices, execution fidelity, and interpretation quality.
• Ensure tasks reflect the nuance and complexity of advanced day-to-day research work rather than simplified textbook exercises.
• Collaborate asynchronously with project leads and reviewers to refine tasks, data, and evaluation frameworks.
• Maintain high standards for methodological rigor, detail, transparency, and reproducible research practices.

Qualifications

• Bachelor''s or Master''s degree in sociology, economics, psychology, political science, or a related social science discipline.
• At least 2 years of hands-on experience designing survey instruments, coding qualitative data, conducting literature reviews, and using statistical software such as SPSS, R, or Stata.
• Strong source-synthesis skills and the ability to create well-documented data artifacts.
• Demonstrated research mastery, methodological rigor, and attention to rubric fidelity.
• Experience developing or applying comprehensive grading rubrics or evaluation guidelines.
• Comfort with detailed, methodical, and occasionally repetitive assignments involving multiple data types and sources.
• Proficient written English for producing technical research content and evaluation materials.

Work Terms

• Remote, independent contractor engagement.
• Compensation is output-based and paid for each task that meets project specifications. Completion time may vary based on experience and workflow.
• Minimum submission requirements apply, including a required minimum number of tasks per week.
• Roles are typically filled within 48 hours. Selected applicants should be ready to begin their first tasks within 24 to 48 hours after completing onboarding.

Compensation

$30 to $50 per hour.

Application Process

Submit an application using an email address or Google account. Applicants who are selected complete onboarding before beginning work.

Job Tags

Similar Jobs