The loss of tropical forests has continued in recent decades despite wide recognition of their importance to maintaining biodiversity. Here, we examine the conversion of forests to pastures and coca crops (illicit activity) on the San Lucas Mountain Range, Colombia for 2002–2007 and 2007–2010. Land use maps and biophysical variables were used as inputs to generate land use and cover change (LUCC) models using the DINAMICA EGO software. These analyses revealed a dramatic acceleration of the pace of deforestation in the region, with rates of conversion from forest to pasture doubling from the first to the second period. Altitude, distance to other crops, and distance to rivers were the primary drivers of deforestation. The influence of these drivers, however, differed markedly depending on whether coca cultivation or pastures replaced forest. Conversion to coca was more probable farther from other crops and from settlements. In contrast, proximity to other crops and to settlements increased conversion to pasture. These relationships highlight the different roles of coca and pastures in forest loss, with coca tending to open up new forest frontiers, and pastures tending to consolidate agricultural expansion and urban influence. Large differences between LUCC processes for each period suggest highly dynamic changes, likely associated with shifting underlying causes of deforestation. These changes may relate to shifts in demand for illicit crops, land, or mining products; however, the data to test these hypotheses are currently lacking. More frequent and detailed monitoring is required to guide actions to decrease the loss of forest in this highly vulnerable biodiversity hotspot in the Northern Andes.