Vtyjkyuk

In proving that beginalign limlimits_nto infty left[int^b_af^n(x)dxright]^frac1n=Mendalign
where $M=max f(x):xin[a,b]$ and $f:[a,b]toBbbR$ is nonnegative and continuous.$

If $f(x)leq M,;forall,xin[a,b]$, I know that $(f(x))^nleq M^n,;forall,xin[a,b]?$ but do we have that $f^n(x)leq M^n,;forall,xin[a,b]?$

By $f^n(x)leq M^n,;forall,xin[a,b]$, I mean "composition in $n$ times". So, in essence, I'm asking: Is beginalign(f(x))^n=f^n(x),;forall,xin[a,b]?endalign Thanks!

First of all, let me point out that the conditions of $f$ is very problematic. To make sense of multiple composition of $f$, you require the range of $f$ is in $[a,b]$. But, if the interval $[a,b]$ lies completely in the negative region, say $a=-2$, $b=-1$, you cannot simultaneously have that $f$ is nonnegative.

Now, let's define an $f$ that does not meet such difficulty. Define $f(x)=0.5$ for all $xin[0,1]$. It meets all the requirements: the range lies in $[0,1]$, multiple composition is well-defined, and the function is positive and continuous.

We have $f(x)=0.5le0.5$, but $f^n(x)=0.5gt0.5^n$ for all $nge2$.

As you may notice, it is even a counterexample to what you want to prove, because $$limlimits_nto infty 0.5^frac1n=1not=0.5.$$

However, I think you may have misunderstood the theorem you want to prove. The correct formula you want to prove might be $$limlimits_nto infty left[int^b_a(f(x))^ndxright]^frac1n=maxf(x):xin[a,b].$$ This is a well-known fact about $L^p$ norm of functions. You may want to try to prove this "correct" version of the formula.

As edm points out, if you want to talk about $f(f(x_0))$ for $f : left[ a, b right]longrightarrow mathbbR$, you need to be sure that $f(x_0) in left[a ,b right]$.

1) If this is not the case, $f(f(x_0))$ doesn't make sense.

2) If this is the case, then since $f(x) leqslant M$ for all $x in left[ a,b right]$ this inequality holds in particular for $x = f(x_0)$, meaning that $f(f(x_0)) leqslant M$. However, we don't necessarily have $f(f(x_0)) leqslant M^2$ since if $0 < M < 1$, $M^2 < M$.

EDIT: I see that you're also asking whether $f^n (x) = (f(x))^n$ holds identically. It doesn't in general: If $f$ is a linear function, then the composition $f^n (x)$ is linear, whereas $(f(x))^n$ is a polynomial of degree $n$.