by Janos Kramar 824 days ago | Patrick LaVictoire likes this | link | parent In order to understand what the measure $$\mu$$ that was constructed from $$d$$ will reward, here’s the sort of machine that comes close to $$\sup_M\mu(M)=3$$: Let $$M_0$$ be an arbitrary UTM. Now consider the function $$r(n)=n-2^{\lfloor \lg n \rfloor}$$ (or, really, any function $$r:\mathbb{N}^+\rightarrow\mathbb{N}^0$$ with $$r(n)2,x_{|x|-1}=x_{r(|x|-1)},x_{|x|-2}=x_{r(|x|-2)}\}$$. (The indices here are zero-based.) Choose $$x_0\in L$$ such that $$x_0$$ has no proper prefix in $$L$$. Then, construct the UTM $$M$$ that does: repeat: s := "" while s not in L: # if there is no next character, halt s := s + readchar() if s == x0: break M0() This $$M$$ will have $$\mu(M)>3-2^{-|x_0|}+d(M_0,M)2^{-|x_0|-d(M_0,M)}$$. $$M$$ here is optimized for building up internal states (that are then UTMs that are efficiently encoded), while also being very easy to reset from these internal states; in other words being easy to “encode” from the UTMs it efficiently encodes, using at most 2 bits (an average of $$\frac{1+\sqrt{5}}{2}$$). This is somewhat interesting, but clearly doesn’t capture the kind of computational expressivity we’re primarily interested in.

### NEW DISCUSSION POSTS

Note that the problem with
 by Vadim Kosoy on Open Problems Regarding Counterfactuals: An Introd... | 0 likes

Typos on page 5: *
 by Vadim Kosoy on Open Problems Regarding Counterfactuals: An Introd... | 0 likes

Ah, you're right. So gain
 by Abram Demski on Smoking Lesion Steelman | 0 likes

> Do you have ideas for how
 by Jessica Taylor on Autopoietic systems and difficulty of AGI alignmen... | 0 likes

I think I understand what
 by Wei Dai on Autopoietic systems and difficulty of AGI alignmen... | 0 likes

>You don’t have to solve
 by Wei Dai on Autopoietic systems and difficulty of AGI alignmen... | 0 likes

Your confusion is because you
 by Vadim Kosoy on Delegative Inverse Reinforcement Learning | 0 likes

My confusion is the
 by Tom Everitt on Delegative Inverse Reinforcement Learning | 0 likes

> First of all, it seems to
 by Abram Demski on Smoking Lesion Steelman | 0 likes

> figure out what my values
 by Vladimir Slepnev on Autopoietic systems and difficulty of AGI alignmen... | 0 likes

I agree that selection bias
 by Jessica Taylor on Autopoietic systems and difficulty of AGI alignmen... | 0 likes

>It seems quite plausible
 by Wei Dai on Autopoietic systems and difficulty of AGI alignmen... | 0 likes

> defending against this type
 by Paul Christiano on Autopoietic systems and difficulty of AGI alignmen... | 0 likes

2. I think that we can avoid
 by Paul Christiano on Autopoietic systems and difficulty of AGI alignmen... | 0 likes

I hope you stay engaged with
 by Wei Dai on Autopoietic systems and difficulty of AGI alignmen... | 0 likes