Greater investment in agriculture is needed to reduce rural poverty and improve food security; but how investment is made, its context and conditions, is at least as important as how much is invested. This paper presents three case studies of large-scale agricultural investment in Paraguay, Guatemala, and Colombia and shows how monoculture expansion is displacing communities, undermining smallholder livelihoods, and worsening local food security. In some cases displacement is a direct result, when companies acquire land from smallholders. In others it is indirect, when smallholders next to plantations are unable to coexist with the health and environmental problems caused by the intensive use of agrochemicals. Cases also showed how large-scale monoculture expansion is competing for land with small-scale basic food production; thus, households which used to be self-sufficient in food now rely on local markets. Even when companies claim to operate responsibly, their business model determines who bears the risks, who has access to capital, and where market power lies. Responsibility should mean benefits and costs are fairly distributed and all rights upheld, including land rights. Private agricultural investment is needed, but it should complement rather than undermine smallholders, who are the main investors in agriculture.