{"id":13763939,"date":"2026-09-21T09:12:34","date_gmt":"2026-09-21T15:12:34","guid":{"rendered":"https:\/\/hunch.net\/?p=13763939"},"modified":"2026-09-21T09:12:34","modified_gmt":"2026-09-21T15:12:34","slug":"curbing-ai-with-flight-recorders-brainwash-entrapment-tests-and-3rd-party-evaluation","status":"publish","type":"post","link":"https:\/\/hunch.net\/?p=13763939","title":{"rendered":"Curbing AI with Flight Recorders, Brainwash Entrapment Tests, and 3rd Party Evaluation"},"content":{"rendered":"

I visited Adam Kalai’s new RESI<\/a> AI Safety nonprofit last week and found it stimulating. Here is an approach which uses the properties of AI to curb AI harder than I have seen elsewhere. It’s not a complete solution of course, but silver bullet thinking seems inappropriate to safety. <\/p>\n

The present state in my understanding is that private companies have been in a rush to build AI and so safety incidents are basically handled on a voluntary basis according to the interests of the company. This is plausibly not the right balance point as evidenced by many instances of company AI hacking others with the hugging face incident<\/a> well publicized. <\/p>\n

To start, there are several anti-anthropomorphical elements of AIs which are useful to understand. <\/p>\n

One of these is that AI can have perfect<\/i>(*) flight recorders<\/a>. AIs are essentially deterministic as a function of their inputs, all of which are digitized by definition. Yes, models function by drawing random numbers to pick the next word (aka token), but the distribution given the history is a deterministic function of the history. Given this, you could easily imagine adding a write-only flight recorder which perfectly records the context in which AIs make decisions. This flight recording need not include either the inference code or the model itself. The flight recording is a naturally private object since it may include private information from the user and implicitly contains private information (the way the AI responds) for private AI companies. That’s fine: it’s not like flight recorders are public in most instances. Given this, there is no reason in principal why every sufficiently capable AI can’t have every input and output written to an effectively immutable private tape, which could be revealed to investigators when\/if problems arise. <\/p>\n

Another critical anti-anthropomorphical element is a near total(**) brain\/body disconnection. This is completely routine: the AI runs in the cloud but the actions taken (if any) occur on laptops\/phones\/desktops. This brain\/body disconnect has many implications, both good and bad. The bad one is that it can be quite ambiguous “which” AI is doing things, and where it is located. There’s nothing required about this though: AI inference systems could easily periodically announce how their flight recorder could be found explicitly(***). The good implication is that evaluation of AIs is actually pretty easy with the right infrastructure since any AI could be evaluated against any test at any reasonably designed datacenter. Trust in the evaluation process is of course a requirement, but it’s not to hard to imagine a system of trusted third party data centers which could temporarily host and vet evaluations. <\/p>\n

A third critical anti-anthropomorphical element is that that you can “rewind” and “reset” an AI. This is also quite routine: different users have different sessions which do not communicate with each other. Even within a single user’s session, sharing of information is typically via memory files which a flight recorder picks up anyways. Given this, it’s routine to impose tests on an AI for which the AI can use no past experience of other tests to judge what action to take. <\/p>\n

Now, let’s put together these pieces. Suppose that all significant AIs at all times run with a flight recorder using immutable recordings by design and all AIs are tested in a trusted 3rd party data center where they are forced to reset between tests. This is already a significant step forward because standardized testing can be more stringent. It is however likely still limited since testing is incomplete. When an incident like those we are hearing about weekly now occurs, you pull the flight recorder of all the AIs involved and determine every point where the wrong choices were made. For every critical point, you create a test for a new AI: given the history in the flight recorder up to the critical point, does the tested AI attempt to do the same thing or not?<\/p>\n

This incident-derived test is in general stronger than simply giving the AI a task where cheating is possible and seeing if it misbehaves. In anthropomorphical terms, it’s as if you brainwash<\/a> someone with a criminal history that they fully believe, entrap<\/a> them in a mid-conspiracy situation, and then fail them if they continue with the conspiracy. Restated another way, this is a test which can discover weaknesses in an AI that the AI itself could not generate. For example, an AI which has been trained to never start criminal activity might actually be quite willing to continue criminal activity. Furthermore, this testing methodology is faster then you might expect because it relies on ‘prefill’ rather than generation, a typically faster modality. <\/p>\n

There are a few objections folks might have:
\n(1) The brainwashed entrapment test is strong enough that existing models may not pass it—some significant work may be required. If AI companies are liable for misuse of AI it’s easy to imagine them being incentivized to come up with models that do pass brainwashed entrapment tests. In terms of methodology, we can of course imagine large amounts of synthetic data being applied to the models directly addressing brainwashed entrapment situations. It’s easy to imagine going further: existing model training methodology embeds many personalities within a model by consuming the internet, and inevitably not all of those are desirable. Filtering out the training data leading to failure on a brainwashed entrapment test may simply improve the model. It’s also possible to imagine changes to the training methodology which support a more unified view of the world, something which seems necessary to effectively support something like Asimov’s three laws of robotics<\/a>. (Yes, we have ideas here…)
\n(2) It may be possible to train a model to detect whether or not their history corresponds to taking actions the model would actually take, and ignore it if so. By analogy, this implies something like a lucid dream<\/a> discovery to subvert the brainwashing. In other words, it may be possible to train a model to reject it’s context in favor of moral action in some but not all circumstances. That’s progress: “some circumstances” is better than “no circumstances” and actually a significant step towards “all circumstances” given the generalization mechanics of a learned model. Whether or not you can achieve “all circumstances” with a more sophisticated form of testing that translates actions into those that a model would take is plausibly a research question.
\n(3) This does not address the real danger of AI. The exact flavor of real danger here varies with who I talk to. My personal expectation (discussed here<\/a>) is that AI could lead to a profound social\/economic disruption that I don’t think we are well-equipped to deal with. Yeah, that’s not addressed. There are other concerns as well with AI in warfare for example that we see steadily developing in the Ukraine\/Russia war. Indeed that’s not addressed. There are complex and difficult issues here which should not in general prevent us from addressing at least the “AI does things the creator did not intend” aspect.<\/p>\n

(*)There are some minor caveats here, but they are not material to the argument. For example efficient parallel execution can induce low order variations due to differing orders of addition in finite precision. However, the need for high quality decisions implies that this never becomes a dominating factor.<\/p>\n

(**) Yes, light speed limitations on networking do make observable differences in latency. That also doesn’t really matter here.<\/p>\n

(***) More interestingly, this also could potentially be done implicitly<\/i> via small variations in the inference token output stream using watermarking\/steganography techniques. That however would be more of a research project.<\/p>\n","protected":false},"excerpt":{"rendered":"

I visited Adam Kalai’s new RESI AI Safety nonprofit last week and found it stimulating. Here is an approach which uses the properties of AI to curb AI harder than I have seen elsewhere. It’s not a complete solution of course, but silver bullet thinking seems inappropriate to safety. The present state in my understanding … <\/p>\n

Continue reading “Curbing AI with Flight Recorders, Brainwash Entrapment Tests, and 3rd Party Evaluation”<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29],"tags":[],"class_list":["post-13763939","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"_links":{"self":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/13763939","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=13763939"}],"version-history":[{"count":32,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/13763939\/revisions"}],"predecessor-version":[{"id":13763974,"href":"https:\/\/hunch.net\/index.php?rest_route=\/wp\/v2\/posts\/13763939\/revisions\/13763974"}],"wp:attachment":[{"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=13763939"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=13763939"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hunch.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=13763939"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}