Have you correctly configured the initial configuration for your PHY? Most PHYs use the signals normally driven by the PHY to capture the initial configuration, using pull-up and pull-down resistors. The TI PHY used on the AM65x/DRA80x EVM uses both to generate a mid-level voltage, allowing four separate configurations per pin. By default in the AM65x/DRA80x, both the receiver buffers and the internal pulling resistors are disabled at reset. The PHYs should be removed from reset, allowing the initial configuration to be captured before enabling either the receiver buffers or the internal pulling resistors.
Have you terminated your RGMII signals correctly? For the RGMII interfaces, 22-Ω series termination resistors must be placed on all 12 interface signals as close to the transmitter as possible.