Intelligent systems in accounting, finance and managementassessing predictive performance of ann-based classifiers, страница 16

As Table IV shows (all significance coefficients equal 0.000), all the pairs of training accuracy rates vectors are statistically different. The direction of the difference is given by the statistics calculated. The Mann–Whitney U statistic corresponds to the better group, in the sense that it represents the smaller number of cases with higher ranks between groups. The Wilcoxon W statistic is simply the smaller of the two rank sums displayed for each group in the rank table. The Kolmogorov–Smirnov Z test statistic is a function of the combined sample size and the largest absolute difference between the two cumulative distribution functions of the two groups. Consequently, by analysing both the calculated statistics and the rank table we can determine the direction of the difference between the groups. For this particular experiment the rank table shows that the accuracy rates are always higher in the case of GA-based ANN training than for RT-based ANN training, thus validating the first hypothesis.

As for training, GA-based ANN training models performed better than gradient-descent-like models in testing (Table V) for all possible GA technique –RT technique combinations.

6.2.  Experiment 2

Our second experiment validates the second hypothesis using non-parametric tests. We preprocessed the real data using normalization and compared the results with those obtained for unpreprocessed data (Table VI). For each combination of the two preprocessing approaches and the seven training techniques (four GA-based ANN and three RT-based ANN), we calculated means for training and testing accuracy rates.

The preprocessing method had an impact on the both training mechanisms’ performances. However, we found a greater impact on the performance for training (U = 0.000) than for testing (U =

6.000). Also, there is greater confidence in the results obtained for training (level of significance: 0.002) than for testing (level of significance: 0.02). Nevertheless, according to the rank tables, we obtained higher accuracy rates when we preprocessed the data using normalization than the case when we used no preprocessing for both training and testing.

6.3.  Experiment 3

To test our third hypothesis we applied the methodology on the fictive datasets and compared the results with those for the real data. Table VII presents the accuracy rates for training and testing samples. For this experiment we used no preprocessing of data. We calculated the means of accuracy rates vectors for each technique–distribution combination.

We applied the non-parametric tests to check the validity of our third hypothesis (Tables VIII and IX). The hypothesis is strongly supported both for training and for testing cases. There is a statistical difference in performance between all distribution pairs except three, namely the real–logistic and uniform–normal pairs in the case of training and the logistic–Laplace pair in the case of testing. The performance order of the distributions fits our expectations: the best accuracy rates were obtained for normally distributed data, followed by data distributed uniformly. The third best performances were achieved for the real dataset, which was superior to the logistic and Laplace distributions in this order.

Table VI. Preprocessing method influence^a