by Sam Eisenstat 877 days ago | link | parent We can also construct an example where $${\rm PA} \vdash \phi \rightarrow \psi$$ with a short proof and $$\rm PA$$ also proves $$\phi \rightarrow \neg\psi$$, but any such proof is much longer. We only need to put a bound on the proof length in A’s proof search. Then, the argument that $${\tt A()} = 1 \wedge {\tt U()} \ne 0$$ proves its own consistency still works, and is rather short: $$O(\log n)$$ as the proof length bound $$n$$ increases. However, there cannot be a proof of $${\tt A()} = 1 \rightarrow {\tt U()} = 10$$ within $${\tt A}$$’s proof length bound, since if it found one it would immediately take action 1. In this case $$\rm PA$$ can still prove that $${\tt A()} = 1 \rightarrow {\tt U()} = 10$$ simply by running the agent, but this argument shows that any such proof must be long.

### NEW DISCUSSION POSTS

When considering an embedder
 by Jack Gallagher on Where does ADT Go Wrong? | 0 likes

The differences between this
 by Abram Demski on Policy Selection Solves Most Problems | 0 likes

Looking "at the very
 by Abram Demski on Policy Selection Solves Most Problems | 0 likes

 by Paul Christiano on Policy Selection Solves Most Problems | 1 like

>policy selection converges
 by Stuart Armstrong on Policy Selection Solves Most Problems | 0 likes

Indeed there is some kind of
 by Vadim Kosoy on Catastrophe Mitigation Using DRL | 0 likes

Very nice. I wonder whether
 by Vadim Kosoy on Hyperreal Brouwer | 0 likes

Freezing the reward seems
 by Vadim Kosoy on Resolving human inconsistency in a simple model | 0 likes

Unfortunately, it's not just
 by Vadim Kosoy on Catastrophe Mitigation Using DRL | 0 likes

>We can solve the problem in
 by Wei Dai on The Happy Dance Problem | 1 like

Maybe it's just my browser,
 by Gordon Worley III on Catastrophe Mitigation Using DRL | 2 likes

At present, I think the main
 by Abram Demski on Looking for Recommendations RE UDT vs. bounded com... | 0 likes

In the first round I'm
 by Paul Christiano on Funding opportunity for AI alignment research | 0 likes

Fine with it being shared
 by Paul Christiano on Funding opportunity for AI alignment research | 0 likes

I think the point I was
 by Abram Demski on Predictable Exploration | 0 likes