Physical victimization and substance use are common behaviors inside prisons. Yet little is known about whether there is an association between substance use and physical victimization among male inmates. We examined the absolute and relative association between substance use (prior to, during, and both prior to and during incarceration) and physical victimization while incarcerated. For this study, 2484 men (mean age of 36.3 years; SD = 11.1) were sampled from eight prisons located in Spain. Information was collected using self-report questionnaires probing experiences of physical victimization while incarcerated and substance use prior to prison and during the past six months in prison. We found that the rates and likelihood of physical victimization were greater for male inmates who used substances at any time, compared to non-consumers of substances but were highest for male inmates who used exclusively in prison or both prior to and in prison. These findings suggest the need for immediate prevention steps including comprehensive screening and segregation practices; better drug interdiction practices; and more evidence-based substance abuse treatment with and without integrated trauma treatment to ensure public health and safety.