The paper introduces a new methodology for assessing on-line the prediction risk of short-term wind power forecasts. The first part of this methodology consists in computing confidence intervals with a confidence level defined by the end-user. The resampling approach is used for this purpose since it permits to avoid a restrictive hypothesis on the distribution of the errors. It has been however appropriately adapted for the wind power prediction problem taking into account the dependency of the errors on the level of predicted power through appropriately defined fuzzy sets. The second part of the proposed methodology introduces two indices, named as MRI and PRI, that quantify the meteorological risk by measuring the spread of multi-scenario Numerical Weather Predictions and wind power predictions respectively. The multi-scenario forecasts considered here are based on the 'poor man's' ensembles approach. The two indices are used either to fine-tune the confidence intervals or to give signals to the operator on the prediction risk, i.e. the probabilities for the occurrence of high prediction errors depending on the weather stability. A relation between these indices and the level of prediction error is shown. Evaluation results over a three-year period on the case of a wind farm in Denmark and over a one-year period on the case of several farms in Ireland are given. The proposed methodology has an operational nature and can be applied to all kinds of wind power forecasting models