On orthogonal and special orthogonal invariants of a single matrix of small order

We study the structure of the algebra of polynomial invariants for the usual conjugation action of the complex special, SO_n, and general, O_n, orthogonal group on the space of traceless n by n complex matrices. (Note that these two algebras coincide if n is odd.) Minimal generating sets of these algebras are known for n less than 5. We construct one for n=5. We also construct a Hironaka decomposition in the case n=3 and a new (more economical) such decomposition for n=4. A simple presentation (with just one syzygy) is obtained for the algebra in the case n=3.