We extend the concept of dual unitary quantum gates introduced in Phys. Rev. Lett. 123, 210601 (2019)PRLTAO0031-900710.1103/PhysRevLett.123.210601 to quantum lattice models in 2+1 dimensions, by introducing and studying ternary unitary four-particle gates, which are unitary in time and both spatial dimensions. When used as building blocks of lattice models with periodic boundary conditions in time and space (corresponding to infinite temperature states), dynamical correlation functions exhibit a light ray structure. We also generalize solvable matrix product states introduced in Phys. Rev. B 101, 094304 (2020)PRBMDO2469-995010.1103/PhysRevB.101.094304 to two spatial dimensions with cylindrical boundary conditions, by showing that the analogous solvable projected entangled pair states can be identified with matrix product unitaries. In the resulting tensor network for evaluating equal-time correlation functions, the bulk ternary unitary gates cancel out. We delineate and implement a numerical algorithm for computing such correlations by contracting the remaining tensors.