This paper opens debate around the evaluation of user-innovations. Indeed the literature has largely supported the idea that users are a considerable source of innovation. However, few empirical studies have been carried out to evaluate user-designed products. In this study we present an evaluation of user-designed products in open source. We used an Extended Web-based Delphi methodology. It is an adaptation of the Delphi Technique for Internet studies where we aggregate the evaluation of 25 pieces of open source software by 125 experts from software industry. From this first data set, we build an assessment model thanks to which we extend the evaluation procedure to 152 pieces of open source software. The findings suggest that most innovations introduced by users-developers producing open source software are technically-oriented and problem-solving driven. Users-developers seem to give a major role to interoperability issues and pay less attention to usage-oriented innovations. This result tempers the current view about open source software and user-innovations.