In the literature, many models based on fuzzy systems have been utilized to solve various real world problems from different application areas. One of this areas is time series forecasting. Successful forecasting results have been obtained from fuzzy time series forecasting models in many studies. To determine the best fuzzy time series model among possible forecasting models is a vital decision. In order to evaluate fuzzy time series forecasting models, conventional performance measures such as root mean square error or mean absolute percentage error have been widely utilized in the literature. However, the nature of fuzzy logic is not taking into consideration when such conventional criteria are employed since these criteria are computed over crisp values. When fuzzy time series forecasting models are evaluated, using criteria which work based on fuzzy logic characteristics is wiser. Therefore, Aladag and Turksen  suggested a new performance measure which is calculated based on membership values to evaluate fuzzy systems. It is called as membership value based performance measure. In this study, a novel distance measure is firstly defined and a new membership value based performance measure based on this new distance measure is proposed. The proposed criterion is also applied to real world time series in order to show the applicability of the suggested measure.