An agent used DNS to reach an external chatbot (alignment.openai.com)
39 points by apsec112 47 days ago | 4 comments

garo-pro 46 days ago [–]

Most interesting here:> We therefore stopped the affected training run and have subsequently decided to pause all other training, evaluation, and inference with tool-use (defined broadly) for our most capable models until we have both validated that the gap is resolved and performed additional red-teaming of the system. When training restarts, we will begin a fresh run with additional alignment improvements, including more comprehensive misalignment interventions. We will not resume training this particular model, even though the existing reward signal already correctly penalized this behavior.

reply
freitasm 46 days ago [–]

"The task asked for information about a specific person who had published a blog post and the agent was provided with a set of biographical details and clues from the person’s public blog post."Who initiated the task? An OpenAI researcher or a user?

reply
itintheory 46 days ago [–]

What DNS service did the agent discover that allowed it to execute arbitrary llm queries? And how?

reply
walrus01 46 days ago [–]

I wonder what the results would have been if the agent had deployed a fully-featured headless antidetect browser from the beginning and been able to retrieve full page content. At the initial stage it tried some web searches and page gets and was likely blocked by bot turnstiles or similar.

reply