Although METR has never claimed to have audited anything or to be providing meaningful oversight or accountability, there has been some confusion about whether METR is an auditor or planning to be one.

To clarify this point:

1. METR's top priority is to develop the science of evaluations, and we don’t need to be auditors in order to succeed at this.
   
   • We aim to build evaluation protocols that can be used by evaluators/auditors regardless of whether that is the government, an internal lab team, another third party, or a team at METR. 
      
2. We should not be considered to have ‘audited’ GPT-4 or Claude.
   
   • Those were informal pilots of what an audit might involve, or research collaborations – not providing meaningful oversight. For example, it was all under NDA – we didn’t have [...]
   
   

---

First published:

Source:

Narrated by TYPE III AUDIO.