Lattice studies of the Sp (4) gauge theory with two fundamental and three antisymmetric Dirac fermions

We consider the Sp(4) gauge theory coupled to Nf=2 fundamental and nf=3 antisymmetric flavors of Dirac fermions in four dimensions. This theory serves as the microscopic origin for composite Higgs models with SU(4)/Sp(4) coset, supplemented by partial top compositeness. We study numerically its lattice realization, and couple the fundamental plaquette action to Wilson-Dirac fermions in mixed representations, by adopting a (rational) hybrid Monte Carlo method, to perform nontrivial tests of the properties of the resulting lattice theory. We find evidence of a surface (with boundaries) of first-order bulk phase transitions in the three-dimensional space of bare parameters (one coupling and two masses). Explicit evaluation of the Dirac eigenvalues confirms the expected patterns of global symmetry breaking. After investigating finite-volume effects in the weak-coupling phase of the theory, for the largest available lattice we study the mass spectra of the lightest spin-0 and spin-1 flavored mesons composed of fermions in each representation, and of the lightest half-integer spin composite particle made of fermions in different representations - the chimera baryon. This work sets the stage for future systematical studies of the nonperturbative dynamics in phenomenologically relevant regions of parameter space.