Friday, 11 November 2016

Judging system idea (credits to Bula)

Bula's suggestion: To have separate judges for each criteria aspect, rather than each judge judging all criteria. For example, taichi and HAL would judge only execution, I would judge only difficulty, fel2fram or RPD would judge only creativity.

For effectiveness and presentation: it is probably fine if all judges graded them, as they do not require as specialised focus/knowledge/high skill level to evaluate, and are not weighted as highly as exec/diff/crea.

Whereas specialisation is old idea applied to many areas of life (medicine, manufacturing etc), this is probably first time I recall it mentioned to me (but I would not be surprised if it was discussed in passing by other spinners before). Nonetheless, credit goes to Bula for stating this idea, when I write this post.

Note: this post discusses his suggestion and related aspects rather than judging in PS in general (as applicability of number scoring to artform like spinning or whether PS can/should even be judged, the deficiencies in judging criteria regarding finesse and fine finger movements for execution, how strictly the original vs creative borderline should be applied, what difficulty is considered 'unbelievable', more than enough for pages and pages etc).

As skill levels in PS (in refinement, technical material, trick/linkage range, difficulty and variations) have increased hugely since typical judging allocations of 'each judge scores all categories' method, which has been used since WT07, specialisation is definitely needed to ensure proper evaluation of various aspects of spinning. Whereas most arts (performance types like music, ice skating etc) or sports do not use specialised judges for different criteria areas, it is quite clear in current PS environment (and has been so since around 2009) that level gap between good spinners' areas of focus compared to areas they do not focus on has increased, and that existing materials shown in PS have also increased enormously.

As spinners focused on creativity probably have better idea of existing concepts, they will have better assessments of creativity and be less likely to think old variations are new. Similarly, spinners who are very focused on technical material aspects probably underestimate the weight of errors in execution or deficiencies in finger positioning/finger movements; whereas spinners who do not focus on using unusual/hard variations are unlikely to be able to separate something that is truly difficult from something that is just a bit hard.

Of course, Bula's system also means each judge has less work to do (as they do not have to judge in all categories), which reduces workload and should increase consistency in scoring within each judge due to lower mental fatigue. As judges selected for each category are specialists/focus on those aspects, there will probably also be less variability in scoring and higher accuracy (i.e. no more menowa WT15 R3 9/10 execution from PSH judge saying combo was dynamic so score was justified, or something like that).

Largest 'problem' with this system is that it ends up narrowing perceptions (reducing open-minded approach to exec/diff/crea) to what the selected group of specialists think, which may decrease innovation. However, I feel small group of specialists in each area should give a better result than having -coughcough- 'strange ideas' contributed from large group of non-specialists.

Another issue with this system is determining which spinners are specialised enough to judge each category (but I feel spinners generally have good idea of who has good control, who is creative, who has hard material - because while individual preferences may be different, community gravitates towards admiring small group of a few dozen or so good spinners who are great at their respective focuses).

There's some spinners who will make combos that will confuse everyone, like fel2fram (who no doubt has top level creativity and difficulty, but his execution is hard to evaluate because we do not really have 'ideal image' that his direction of spinning should appear like), but such out-of-the-box geniuses will always go beyond existing logic.

1 comment:

  1. In 2013, PSHT rules is like this. And it is success.