Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Qualities that alignment mentors value in junior researchers, published by Akash on February 14, 2023 on LessWrong.

This work was performed as a contractor for SERI MATS, but the views expressed are my own and do not necessarily reflect the views of the organization.

I recently conducted interviews with 7 current/former SERI MATS mentors. One of my goals was to understand the qualities that MATS mentors believe are most valuable for junior alignment researchers. I asked questions like:

Who were your most promising scholars? What made them stand out? What impressed you about them?

What are some important qualities or skills that you see missing from most MATS scholars?

What qualities were your scholars most missing? What are some things that you wish they had, or that would’ve made them more impactful?

Qualities that MATS mentors value

Endurance, happiness, & perseverance: Mentors noted that many scholars get discouraged if they’re not able to quickly come up with a promising research direction quickly, or if they explore 1-2 directions that don’t end up being promising. Mentors commented that their most promising scholars were ones who stay energetic/curious/relentless even when they don’t have a clear direction yet.

Hustle + resourcefulness: What do you do when you get stuck? Mentors said that many scholars don’t know what to do when they’re stuck, but their promising mentees were able to be resourceful. They would read related things, email people for help, find a relevant Discord server, browse Twitter, and contact other MATS scholars + AIS researchers for help.

Ability to ask for help + social agency: Many scholars waste a lot of time trying to figure things out on their own. Mentors noted that their most promising scholars were very agentic; they often found other scholars in the program who could help them or other Berkeley researchers who could help them. This also saved mentors time.

Ability to get to know other scholars + engage in peer mentorship: According to mentors, many scholars rarely interacted with others in the stream/program. Some of the best scholars were able to form productive/mutualistic relationships with other scholars.

Strong & concrete models of AI safety: Mentors noted that strong models are important but also hard to acquire. Some mentors emphasized that you often don’t get them until you have talked with people who have good models and you’ve spent a lot of time trying to solve problems. Others emphasized that you often don’t get them until you’ve spent a lot of time thinking about the problem for yourself.

According to one mentor, the best way to get them is just to work closely with a mentor who has these models. No good substitute for just talking to mentors.

Additionally, mentors noted that reading is undervalued. People have written up how they think about things. One mentor said they have read “everything on Paul’s blog, which was super valuable.”

ML and LLM expertise: Some mentors valued ML skills, lots of experience playing around with language models, and strong intuitions around prompt engineering. (Unsurprisingly, this was especially true for mentors whose research interests focused on large language models).

Research communication skills: Being better at efficiently/compactly getting across what they did and what their main problems/bottlenecks were. Some mentors noted that they felt like their (limited) time in meetings with scholars could have been used more effectively if scholars were better at knowing how to communicate ideas succinctly, prioritize the most important points, and generally get better at “leading/steering” meetings.

A few observations

I was surprised at how often mentors brought up points relating to social skills, mental health, and motivation. I used to be a PhD student in clinical psychology, so I was w...