Vtyjkyuk

I'm new to machine learning and one of the things I don't understand about CNN is whether we have to learn the kernel values at every convolutional layer, or just learn a single set of kernel values and use it at every convolution layer.

The answer by @Shehryar Malik is correct (+1), but it sounds a bit confusing, especially for people new to convolutional neural networks.

In the usual CNN scenario, each layer has its own set of convolution kernels that has to be learned. This can be easily seen in the following (famous) image:

The left block shows learned kernels in the first layer. The central and right block show kernels learned in deeper layers¹. This is very important feature of convolutional neural networks: At different layers the network learns to detect stuff at different levels of abstraction. Therefore the kernels are different.

In theory, nothing prevents you from using the same kernels at each layer. In fact, that thing is called recurrent convolutional neural network.

¹ More precisely, they show to what kind of image features these kernels respond to, since visualizing kernel with shape 3$times$3$times$256 is not very easy/intuitive/useful.

That is entirely up to you. You can define only one set of kernel values and use it for all your layers or instead you could define a separate set of kernel values for each layer. Of course, it would be more prudent to define different sets of kernel values for each layer. This is because the kernel's job is to extract specific information from an input image. Different sets of kernel values at each layer will allow the network greater flexibility in deciding the best features to extract at each layer.