Top Potential Future Roles Arc Evals Expressions Of Interest Jobs

Introduction

The evaluations project at the Alignment Research Center is a new team building capability evaluations (and in the future, alignment evaluations) for advanced ML models. The goals of the project are to improve our understanding of what alignment danger is going to look like, understand how far away we are from dangerous AI, and create metrics that labs can make commitments around (e.g. 'If you hit capability threshold X, don't train a larger model until you've hit alignment threshold Y'). You can learn more about the project at this linked post on the Alignment Forum.

We have filled this position, but we may hire again for it in the future. You can express your interest should we open the role back up, but we will not actively review submissions for now and may not ever get back to you.

Job Mission

- Find, evaluate, recruit, manage, incentivize and supervise skilled contractors to generate high quality data on challenging and engaging tasks. Tasks include things like 'using only bash commands, research the most capable open-source language model and set it up on a new server'

- Ensure instructions are clear and incentives are appropriate

- Help contractors feel motivated and part of a team

- Set up training, mentoring or pairing to improve contractor performance

- Understand and fix the biggest bottlenecks to high quality data generation

- Evaluate different data generation contracting services or strategies, and make high level decisions about when to use different out-sourcing services to scale up data generation.

Example Projects

- Manage a team of skilled contractors to extend a small set of expert-produced gold-standard ratings of complex ML trajectories (e.g. 100 examples) into a large dataset, making sure to maintain rating quality close to the gold standard.

- Talk to researchers about how labels should be generated, and then produce clear instructional materials for contractors which help them understand the task.

- Do a small comparison of different contracting services for a data generation task, and decide which, if any, could produce data that meets our bar.

Skills and Experience

- Successfully organized fast-moving projects with many people involved 

- Understanding of LLMs, including finetuning with human data

- Data analytics

- Familiarity with bash, ability to engage with the tasks and produce high-quality data yourself

Introduction

The evaluations project at the Alignment Research Center is a new team building capability evaluations (and in the future, alignment evaluations) for advanced ML models. The goals of the project are to improve our understanding of what alignment danger is going to look like, understand how far away we are from dangerous AI, and create metrics that labs can make commitments around (e.g. 'If you hit capability threshold X, don't train a larger model until you've hit alignment threshold Y'). You can learn more about the project at this linked post on the Alignment Forum.

We expect to keep this posting up, but not necessarily evaluate applications actively at all times. It may be some time (months) before we get back to you.

Job Mission

- Effectively finetune LLMs to demonstrate different behaviors or capabilities

- Make informed guesses about how much a finetuning run will improve model performance at various tasks

- Plan what data we need to collect, and prioritize this given tradeoffs in ease of collecting different types of data

- Structure experiments to rapidly get information about how well our approach is working

- Anticipate and prevent costly mistakes in finetuning efforts. Create and track metrics for finetuning success. 

- Keep up-to-date on the latest techniques, datasets or tools for eliciting or measuring model performance and capabilities, and ensure we're getting an accurate picture of model capabilities

- Design and run experiments in a replicable, scientific manner, producing highly trustworthy results

- Identify novel hypotheses about the capabilities and behavior of large foundation models, and design experiments and data collection strategies to evaluate them.

- Forecast future progress in ML, including how we'll need to change our evaluation strategies, and how we might incorporate future models to speed up our workflows

Skills

- Conceptual alignment thinking: help generate and evaluate ideas for how to probe alignment, deception, agency and other conceptually slippery properties.

- Dealing with the engineering details of large models

- Selecting or improving model architectures

Introduction

The evaluations project at the Alignment Research Center is a new team building capability evaluations (and in the future, alignment evaluations) for advanced ML models. The goals of the project are to improve our understanding of what alignment danger is going to look like, understand how far away we are from dangerous AI, and create metrics that labs can make commitments around (e.g. 'If you hit capability threshold X, don't train a larger model until you've hit alignment threshold Y'). You can learn more about the project at this linked post on the Alignment Forum.

We have filled this position, but we may hire again for it in the future. You can express your interest should we open the role back up, but we will not actively review submissions for now and may not ever get back to you.

Job Mission

- Take ownership of designing and maintaining data pipelines that minimize friction in generating, labeling, quality-checking, filtering, searching and compiling ML datasets covering many aspects of model behavior.

- Take responsibility for the smooth gathering and processing of data at the organization as a whole, and take action to proactively address potential problems.

- Allow researchers to quickly understand properties like quality and balance of the data we've gathered

- Run experiments to discover the most effective prompting or finetuning strategies

- Ensure all results are reliable and easily replicable

- Produce scaling laws and other investigations of how results vary according to model properties

- Keep your eye on the ball of ensuring accurate assessments of model capabilities, be scrappy and willing to move beyond your job description, be part of a fast-moving team.

Example Projects

- Set up an efficient data pipeline and workflow for checking contractor agreement on data labels.

- Set up data pipelines for model behavior trajectories which allow easy searching, filtering and rating.

- Work with different evaluation projects to convert datasets into suitable formats.

- Work with researchers to identify data hypotheses, and set up experiments and visualizations of the data to investigate them

- Work with ML researchers to generate and process data to analyze scaling laws for different model capabilities.

Skills

Essential Skills

- Prior experience with finetuning LLMs or working with natural-language data.

Bonus Skills

- Expertise in particular domains relevant to model takeover, e.g. cybersecurity or ML engineering, in order to help assess model competence on these tasks.

- Design + data-viz skills: Generating novel ideas for improving usability of evaluation tools and providing an interaction experience that unlocks insights about the data or the model

- Experience with large-scale data or distributed systems

- Dealing with the engineering details of large models