This paper is concerned with the smooth and nonsmooth soliton solutions of the Novikov equation based on the bifurcation method of dynamical systems. Two interesting results are highlighted. First, the new Hamiltonian function is established in the case of
$$\varphi ^2c$$
is discussed in Li (Int J Bifurcat Chaos 24(3):1450037 2014). Second, we prove that the corresponding traveling wave system of the Novikov equation exists new smooth and nonsmooth soliton solutions.