NEWxSFC - Interim Summary…as of 25 FEB 2010

AVG SUMSQ

AVG STP

AVG Total Absolute

AVG Absolute

Mean RSQ

 

Previous Ranks

Rank

Forecaster

Class

Total STN 4casts

Error (")

Error Z

% MPRV over AVG

Rank

4cast (")

Error

Error Z

% MPRV over AVG

Rank

Error (")

Error Z

% MPRV over AVG

Rank

Error (")

Error Z

%MPRV over AVG

Rank

RSQ

RSQ Z

% MPRV over AVG

Rank

Forecaster

1,1

1

ilibov

Senior

120

403

-0.903

41%

5

128.7

8.0

-1.110

77%

2

56.67

-0.728

19%

7

2.39

-0.861

25%

6

60.2%

1.089

32%

7

ilibov

3,3

2

donsutherland1

Chief

126

282

-0.894

56%

2

145.8

18.3

-0.640

49%

6

54.25

-1.063

35%

2

2.17

-1.054

35%

2

70.8%

0.988

37%

2

donsutherland1

2,2

3

Shillelagh

Senior

104

523

-0.691

24%

8

107.9

35.7

0.339

-23%

14

61.56

-0.832

19%

7

2.90

-0.496

13%

9

65.4%

0.821

24%

7

Shillelagh

5,6

4

millersville_bauers

Rookie

120

421

-0.667

30%

6

130.7

30.3

-0.051

7%

10

66.69

-0.376

11%

9

2.71

-0.518

15%

7

58.0%

0.538

19%

6

millersville_bauers

6,4

5

herb@maws

Senior

116

440

-0.658

37%

6

143.4

18.4

-0.560

41%

8

61.99

-0.650

21%

5

2.72

-0.533

18%

6

59.4%

0.412

15%

6

herb@maws

10,9

6

Donald Rosenfeld

Senior

126

438

-0.457

31%

6

127.5

25.9

-0.211

15%

7

60.54

-0.715

22%

4

2.42

-0.769

24%

4

54.6%

0.393

17%

5

Donald Rosenfeld

9,10

7

weatherT

Intern

126

352

-0.430

26%

6

99.3

40.5

0.088

-6%

10

61.76

-0.314

14%

6

2.42

-0.518

18%

6

55.5%

0.567

22%

6

weatherT

12,11

8

TQ

Senior

118

400

-0.182

15%

9

97.3

31.3

-0.219

13%

9

65.46

-0.189

9%

8

2.77

-0.083

6%

9

45.1%

-0.172

-1%

9

TQ

4,5

9

Raven

Senior

116

571

-0.036

-9%

7

160.5

31.1

0.079

8%

10

74.39

0.060

2%

9

3.23

0.197

-1%

10

59.0%

0.276

12%

6

Raven

11,12

10

Mitchel Volk

Senior

117

781

-0.014

2%

13

158.8

42.6

0.333

-23%

14

83.40

-0.312

4%

11

3.53

-0.171

2%

12

54.9%

0.002

0%

13

Mitchel Volk

15,13

11

Roger Smith

Rookie

131

711

0.166

-13%

10

202.8

66.1

0.769

-71%

12

98.64

0.408

-15%

11

3.75

0.229

-10%

10

59.6%

0.298

13%

8

Roger Smith

13,14

12

snowman

Senior

116

953

0.584

-19%

15

116.2

35.4

0.325

-21%

12

95.63

0.279

-9%

13

4.13

0.479

-13%

14

37.9%

-1.012

-32%

16

snowman

 

 

There have been seven (7) snowstorm forecasting Contests to as of 25-FEB-10.  Under the ‘two-thirds’ rule…forecasters who have entered at least five (5) forecasts are included in this interim summary.

 

To qualify for ranking in the Interim and final ‘End-of-Season’ standings…a forecaster must enter at least two-thirds of all Contests.  If a forecaster has made more than enough forecasts to qualify for ranking…only the lowest SUMSQ Z-scores necessary to qualify are used in the computing the average.  IOW…if you made nine forecasts…only your six best SUMSQ Z-scores are used to evaluate your season-to-date performance.  You can think of it as dropping the worse quiz score before your final grade is determined.  The reason we have this rule is to 1) make it possible to miss entering a forecast or two throughout the season and still be eligible for Interim and ‘End-of Season’ ranking and 2) encourage forecasters to take on difficult and/or late-season storms without fear about how a bad forecast might degrade their overall 'season-to-date' performance score(s).

 

The mean normalized ‘SUMSQ error’ is the Contest/s primary measure of forecaster performance.  This metric measures how well the forecaster/s expected snowfall 'distribution and magnitude' for _all_ forecast stations captured the 'distribution and magnitude' of _all_ observed snowfall amounts.  A forecaster with a lower average SUMSQ Z Score has made more skillful forecasts than a forecaster with higher average SUMSQ Z Score.

 

The 'Storm Total Precipitation error’ statistic is the absolute arithmetic difference between a forecaster/s sum-total snowfall for all stations and the observed sum-total snowfall.  This metric…by itself…is not a meaningful measure of skill…but can provide additional insight of forecaster bias.

 

The 'Total Absolute error' statistic is the average of your forecast errors regardless of whether you over-forecast or under-forecast.  This metric measures the magnitude of your errors.

 

The 'Average Absolute Error' is the forecaster/s ‘Total Absolute Error’ divided by the number of stations where snow was forecast or observed.

 

The ‘RSQ error’ statistic is a measure of the how well the forecast captured the variability of the observed snowfall.  Combined with the SUMSQ error statistic…RSQ provides added information about how strong the forecaster/s ‘model’ performed.