In this study we analyse the impact of weather forecasts provided to smallholder maize farmers through mobile phone short message service on self-reported labour costs, crop yield and income. We conducted a pilot field experiment, involving 331 randomly selected eligible farmers in six villages. Randomisation was done at the village level. We used three regression specifications to estimate the impacts: Ordinary Least Squares (OLS), Generalised Estimating Equations (GEE) with a small sample correction and Randomisation Inference (RI). We found that the treatment and control groups were well balanced. Farmers in the treatment group recorded lower labour costs but higher crop yield and income levels. Both the direction and the magnitude of the impact estimates were consistent across the three regression specifications, but significant with the RI model only (for labour costs and yield) or the RI and GEE models (for income). Weather forecasts can have an impact on smallholder farmers’ labour, yield and income. These findings are strong evidence of the possibility of using weather-related information and mobile phones to build smallholder farmers’ resilience to climate variability. Yet more research is required to build a solid evidence base to inform agricultural policies.