Intelligent Agent Foundations Forumsign up / log in
by Patrick LaVictoire 981 days ago | link | parent

I can prove the property that for each hypothesis \(A()=a\) there is at most one \(u\) such that \(U()=u\) has a high valuation (for sufficiently high PA+N), with the following caveat: it can sometimes take many steps to prove that \(u\neq u'\) in PA+N, so we’ll need to include the length of that proof in our bound.

In what follows, we will take all subscripts of \(d\) and \(\nu\) to be \(PA+N, A()=a\) for \(N\) large.

For any \(\phi\), \(d(\bot) - d(\neg\phi)\leq d(\phi)\leq d(\bot)\) and thus \[1 - \frac{d(\phi)}{d(\bot)} \leq \nu(\phi) \leq \frac{d(\bot)}{d(\phi)+d(\bot)}.\]

Also, \(d(U()=u)+d(U()=u')+d(u\neq u')\geq d(\bot)\). This implies \(\max\{d(U()=u),d(U()=u')\}\geq \frac12(d(\bot)-d(u\neq u))\), which implies \[\min\{\nu(U()=u),\nu(U()=u')\}\leq \min\{\frac{d(\bot)}{d(U()=u)+d(\bot)},\frac{d(\bot)}{d(U()=u')+d(\bot)}\} \leq \frac{2d(\bot)}{3d(\bot)-d(u\neq u')}.\]

So we see that \(\nu(U()=u)\) and \(\nu(U()=u')\) cannot both be significantly larger than 2/3 if there is a short proof that \(u\neq u'\).





Note: I currently think that
by Jessica Taylor on Predicting HCH using expert advice | 0 likes

Counterfactual mugging
by Jessica Taylor on Doubts about Updatelessness | 0 likes

What do you mean by "in full
by David Krueger on Doubts about Updatelessness | 0 likes

It seems relatively plausible
by Paul Christiano on Maximally efficient agents will probably have an a... | 1 like

I think that in that case,
by Alex Appel on Smoking Lesion Steelman | 1 like

Two minor comments. First,
by Sam Eisenstat on No Constant Distribution Can be a Logical Inductor | 1 like

A: While that is a really
by Alex Appel on Musings on Exploration | 0 likes

> The true reason to do
by Jessica Taylor on Musings on Exploration | 0 likes

A few comments. Traps are
by Vadim Kosoy on Musings on Exploration | 1 like

I'm not convinced exploration
by Abram Demski on Musings on Exploration | 0 likes

Update: This isn't really an
by Alex Appel on A Difficulty With Density-Zero Exploration | 0 likes

If you drop the
by Alex Appel on Distributed Cooperation | 1 like

Cool! I'm happy to see this
by Abram Demski on Distributed Cooperation | 0 likes

Caveat: The version of EDT
by 258 on In memoryless Cartesian environments, every UDT po... | 2 likes

[Delegative Reinforcement
by Vadim Kosoy on Stable Pointers to Value II: Environmental Goals | 1 like


Privacy & Terms