We investigate the nonlinear evolution of pairs of three-dimensional, equal-sized and opposite-signed vortices at finite Froude and Rossby number. The two vortices may be offset in the vertical direction. The initial conditions stem from numerically obtained relative equilibria in the quasi-geostrophic regime, for vanishing Froude and Rossby number. We first address the linear stability of the quasi-geostrophic opposite-signed pairs of vortices and show that for all vertical offsets, the vortices are sensitive to an instability when close enough together. In the nonlinear regime, the instability may lead to the partial destruction of the vortices. We then address the nonlinear interaction of the vortices for various values of the Rossby number. We show that as the Rossby number increases, destructive interactions, where the vortices break into pieces, may occur for a larger separation between the vortices, compared to the quasi-geostrophic case. We also show that, for well-separated vortices, the interaction is non-destructive and ageostrophic effects lead to the deviation of the trajectory of the pair of vortices, as the anticyclonic vortex dominates the interaction. Finally, we show that the flow remains remarkably close to a balanced state, only emitting waves containing negligible energy, even when the interaction leads to the destruction of the vortices.