Halliday scores and stats

[Croquet King]

In the latest Halliday magazine he's written an article about scores and judging, in particular in reference to the 2016 Wine Companion.

From the stats he's provided:
There are 8958 wines in the 2016 book.
76% scored 94 points or less
28% scored 89 points or less

so this means that in terms of bracketing the scores

100 to 95 - 24% or 2150 wines
94 to 90 - 48% or 4300 wines
89 or less - 28% or 2508 wines

While I appreciate that not all wines are tasted for inclusion in the book, a lot are.
We make a lot of good wine in Australia but I just can't comprehend that almost a quarter of them are rated 95 points and above.
Maybe it would be interesting to find out how many were tasted but never made it into the book.

Either way these are the hard numbers he's provided.

[michel]

couldnt agree more re the volume of 95 plus wines
i dont ready any of his reccos these days

[ufo]

I do have a huge respect for him for who he is and what he has done for AU wine industry but he is making a fool of himself with his scores and it is getting worse.

[Redwine&Rum]

Yeah I don't really pay all that much attention. I normally take 2-3 points off his ratings anyway.

Which wine reviewers out there do you feel are on the money with their ratings and are not inflated. In fact, I wouldn't mind reading someone's reviews that sometimes may be a little harsh, as that means that their top scores are going to be incredible wines.

Anyone point me in the right direction here?

[swirler]

He's rebranded everything Halliday, now, which is interesting as it ties the products closer to him (increases the value of the brand?)

His scores are becoming more and more predictable and meaningless. He's prepared to stick the boot into the small players, but is very kind to the corporates (he was a corporate lawyer.)

Still people keep buying his top wines. This year's one goes for $400+ at auction.

[swirler]

Yeah I don't really pay all that much attention. I normally take 2-3 points off his ratings anyway.

Which wine reviewers out there do you feel are on the money with their ratings and are not inflated. In fact, I wouldn't mind reading someone's reviews that sometimes may be a little harsh, as that means that their top scores are going to be incredible wines.

Anyone point me in the right direction here?

I think it is folly to buy wines based on others' opinions. Buy a bottle and taste it yourself before committing to a big buy. Go to tastings if you live in a big city. Don't get sucked into the hype of big score wines. That 'must have' wine will be available somewhere in the future if you keep your eyes peeled. And you'll probably be disappointed when you eventually taste it.

[WineRick]

Interesting quote from Hugh Johnson, speaking 20 years ago at the Sydney wine show.

"I judge wines by loving it or hating it and there's not much in between. I love vitality in a wine, the sort of wine where one bottle is not enough. So giving wines points creates a spurious sense of accuracy and if you can believe it means something when someone gives a wine 87 points out of 100, then you would believe anything."

Another trend happening with the Hardie Grant/ Halliday brand is the difference in scores depending on who tastes it. Within the 'stable' there has been significant variation in scores on the very same wine. A number of years ago, Halliday 'sniped' in an edition of his Companion that North American readers should add 5 points to his scores. Is this still the case. Seems different now!
I guess to give high points means the clearing houses, and ads in the press generally, will quote you often, which in turn helps the brand. Hardie Grant has been very clever by now branding the Companion simply 'HALLIDAY', so anyone quoting points can reference Halliday, unlike The Advocate, where the impact of scores for wines tasted by Lisa P-B is nowhere near as great as quoting 'Parker'.

[Redwine&Rum]

Yeah I don't buy wines purely on others opinions. But I use it in conjunction with a whole host of information when I do.
I live in Alice Springs, so naturally tasting of the wines 9 times out of 10 is out of the question. 90% of the wines I drink are either direct from the winery or from auction sites. And it costs a pretty penny to post stuff here a well.

I use this forum a lot for advice, on what's good, and what is not.

And yes, certainly take Halliday scores with a big pinch of salt. A really good rating might peak my interest and I'll look further into it (winery, vintage history, other reviews, forum conversation on tastings etc).

I don't mind having the odd look at Huon Hooke. I find his reviews are rarely over inflated? But then again, I could be wrong. haha

[Chris H]

He is getting on in years now and I am pretty sure most tasting notes are no longer done by him anyway.

[JamieBahrain]

The goose is cooked.

Great reviews for retailers where they can plaster a cheery note and outrageous score on a flyer.

[Polymer]

A number of years ago, Halliday 'sniped' in an edition of his Companion that North American readers should add 5 points to his scores. Is this still the case

I think this was specifically in reference to the RP Aussie fruit bombs that were making it overseas....but the reason I find it ironic is I think everyone else generally takes 5 points off of JH's score to come up with a more "in line" score for most other things...

In 5 years AU will be on the 2-3 point scale...and have to use tenths of a point...

[catchnrelease]

Which wine reviewers out there do you feel are on the money with their ratings and are not inflated. In fact, I wouldn't mind reading someone's reviews that sometimes may be a little harsh, as that means that their top scores are going to be incredible wines.

Anyone point me in the right direction here?

Mike Bennie.

[swirler]

I'm the only person who nails it each time. Halliday sometimes gets it right (when he agrees with me.) JO sometimes gets it 'right' and CM, too. But they often get it wrong.

Of course, I'm joking as there is no such thing as a right score. Take 100 wines and 100 drinkers and no 2 people will agree 100% with anyone else.

'Experts' are useful when you know nothing. You may find a reviewer who you often agree with, but there will be plenty of times when they disagree.

Life isn't simple. Good job, too, as it would be boring. Why follow another human with all his/her idiosyncrasies when you can plough your own furrow enjoying the journey including the ups and downs.

That's just common sense, isn't it?

[Michael McNally]

Which wine reviewers out there do you feel are on the money with their ratings and are not inflated. In fact, I wouldn't mind reading someone's reviews that sometimes may be a little harsh, as that means that their top scores are going to be incredible wines.

Anyone point me in the right direction here?

Mike Bennie.

I think any reviewer has to have their "scores" assessed over time.

Case in point. I agree that Mike Bennie seems more critical or frank in his assessment of "commercial" wines. However, I think his scores on "natural" or "quirky" or "garagiste" wines seem to reflect his preference/appreciation of these wines. Unsurprising!

This is not to say he is biased, just that the scores might reflect how he assesses the wine overall, which cannot help but be affected by what he sees as good/worthy wines.

Other writers will note that this wine is "not their cup of tea" and rate it highly, as does MB, but it's still a subjective assessment of the wine.

I think the notes are far more important than the scores. In terms of notes I find I tend to agree with Campbell Mattinson's descriptions and assessments of wines, and also often with Halliday (or whoever is doing his notes), but I try to disregard the scores of both - a bit in the case of CM and a lot in the case of JH.

Cheers

Michael

[rens]

When I see a Halliday rating I normally don't take any notice of the score. Unless its 98, 99 or 100-Then I may take notice and see what others think. Gary Walsh is not a bad critic and I like his opinion as well as Jeremy Oliver for his assessments (2000 Grange anyone?).
I see the wine critic as having a specific job-to be critical of the wine, not as some 'critics' see their job as a sales person of that wine.
I note Jeremy Oliver has never been invited back to a vertical Grange tasting by Penfolds since he slammed the 2000 Grange.

[sjw_11]

Ah this topic once again, always such fun.

Are Halliday's scores normally distributed if you allow for all the wines he doesn't put in the book and assume scores can only be 70pts - 100pts?
So Halliday's website claims he has reviews for 2,500 Aussie wineries (although Winetitles directory only has about 1,900)... but assuming that is current and each winery produced 10 wines on average... then you might imagine the total pool to sample was 25,000

Which suggests that the 2,150 wines rated 95-100 could be ~8.6% of the conceptual total pool of wines that either did or did not feature in the book...

Question: do we generally agree that ~10 wines per winery is right? I am thinking maybe too much across all 2,500 wineries as that will doubtless include 100s of very very small producers... but then again you might have Penfolds making 200 wines?

However, if that analysis is right, then we could further say that in reality the scoring range for non-faulty commercial made wine is 70-100 points.

Using that as the scale and assuming scores were in fact normally distributed around a mean of 85 then I reckon you would expect 5% of wines to score over 95? (someone else may have more correct maths)

So Halliday would be a bit high, but not extravagantly so.

Similarly, his points would imply 25% of all the wine in Australia is 90-100 points... which, if you only use 70-100pts as a scale, again feels a little high but probably not massively so (maybe you could say his points are 5% too high versus a totally random normal distribution?)

[felixp]

Halliday and his notes/scores have become a complete joke. Only useful for merchants to try and sell rubbish wine. Internationally, he is laughed at, the Honkies think he is a big joke. Really sad, he has destroyed what was once a brilliant career and reputation.

[Polymer]

He hasn't destroyed his career or reputation..he has a great reputation in Australia...just that no one cares about his scoring anywhere else.

Doesn't matter..the only points are I care about are Wheelers..

[JDSJDS]

I do have a huge respect for him for who he is and what he has done for AU wine industry but he is making a fool of himself with his scores and it is getting worse.

+1

From one outsider's perspective, the worst thing about all the high scores that Halliday has always thrown out is that almost everybody follows his lead; if they don't they risk not getting any attention. The bottom line is that higher points + more money for producers and a higher visibility and thus more influence for the reviewer. So all high profile critics in Australia seem to be more than 'generous' in their scores. I take off between 5-7 points for all main Aussie critics. The fact that they typically only judge Australian wines makes the marking scheme even more lopsided and ridiculous. The situation is a serious blow to the Australian wine scene, IMO.

The thing is, it didn't have to be this way. Indeed, I would argue that having a 'fair' scorer as a critic, and having an international perspective is far better in the long run for producers, as it provides more reasonable feedback for producers so they are less likely to rest on their laurels. The main Canadian wine critic, A. Gismondi, is a good example. Most Canadian wines score in the low to mid 80s, which is where they belong. Getting a 90 score is pretty rare for a Canadian wine (that reflects the reality I experience too). While he sometimes reads like a grumpy old man, I've always respected NZ's Geoff Kelly for making sure to include international wines to compare with NZ wines, clearly noting high preponderances of flawed wines in some tastings and having a fairly reasonable scoring system. I wonder why Australia ended up with such a biased scoring system? Was it the high prices for non-domestic (besides NZ) wine and thus almost a complete focus on domestic wines? While every country has the non-critical, local 'rah-rah' critics (Raymond Chan [NZ], anyone?), Australia really seems have the highest scoring critics.

[Ozzie W]

There another issue with Halliday which nobody else has mentioned. There are frequently delays in the publishing of his tasting notes. I suppose he has to hold some back so he's got something fresh to publish in his book? But his notes are of little use to me if I can't buy the wines he raves about because they've already sold out. On the contrary, at The Wine Front, the wines are frequently reviewed before they are released to the public.

[bdellabosca]

I see the wine critic as having a specific job-to be critical of the wine, not as some 'critics' see their job as a sales person of that wine.

+ 1

I wish I had the luxury of the time and access to taste every wine before I buy and / or the money and storage to buy lots of wine knowing I will get some outstanding wines in the mix. I don't and I'm guessing that this is the same for most wine buyers. I use critic reviews (in particular scores but also reading notes and selecting based on my style / varietal preferences) and any other online / written reviews (e.g. this forum, cellartracker, etc...) in trying to differentiate wines and drive my purchases.

Whilst not ideal, I can live with the differences in their respective points scale (after time I usually get a feel for each critic, e.g. JH - 3 pts = GW sort of, etc...). My biggest issue is bias - a critic who is not independent and rates certain wines / brands higher for non-quality related reasons (e.g. kick-backs / conflict of interest / publicity / stay in good books / patriotism) is effectively lying / misleading / failing at their job.

IMO there should be an industry standard that requires all wine critics to do tastings and reviews (both notes and scores) with no knowledge (at all! - based on a standardised tasting process guidelines similar to what should be happening for wine shows, etc...) of what wine they are tasting and make a positive statement that they complied with this industry standard in all reviews published (and perhaps restricted from doing wine reviews for a period of 12 months if ever discovered they did not do it fully blind, changed notes after finding out the wine identity, etc...?). What do people think about this as an idea for an industry standard for wine reviews...? Easy to implement / enforce I am sure!!!

[JamieBahrain]

You don't think a Halliday point creep has affected all the commentators in Oz?

[kaos]

Halliday actually has a bizarre 'out' for all of this: his scale is not an absolute scale. It has always irked me that he scores champagne on a different scale than Australian Sparkling (ie a 95pt Champers is not rated same quality and a 95pt Oz bubbles). Once you allow for these vagaries then the whole thing become kind of meaningless. He can give an Oz Pinot Noir 99 pts and then when faced with the complaint that it doesn't hold a canndle to xyz premier cru and he has basically left himself nowhere to go he can use the Oz Pinot 100 point scale defence: all the 99pts means is that it is near the perfect Oz Pinot...... How he determines what constitutes perfection is the question and how does he know that it cannot be improved upon....

[sjw_11]

Halliday actually has a bizarre 'out' for all of this: his scale is not an absolute scale. It has always irked me that he scores champagne on a different scale than Australian Sparkling (ie a 95pt Champers is not rated same quality and a 95pt Oz bubbles). Once you allow for these vagaries then the whole thing become kind of meaningless. He can give an Oz Pinot Noir 99 pts and then when faced with the complaint that it doesn't hold a canndle to xyz premier cru and he has basically left himself nowhere to go he can use the Oz Pinot 100 point scale defence: all the 99pts means is that it is near the perfect Oz Pinot...... How he determines what constitutes perfection is the question and how does he know that it cannot be improved upon....

Except this "out" doesn't work if he still scores the wines systematically too high versus the available universe... which my logic above suggested is the case...

For what its worth, I don't actually object much to this logic though.

[swirler]

I see Winefront scores are now on wine-searcher. Probably nobody outside of Australasia will have ever heard of them before.

[tarija]

Agree with Mike Bennie as being one of the better Australian critics - or rather, his point scoring is probably more closely aligned with international scoring.

Jeremy Oliver is good in that he is extra-conservative. I am happy with this, as opposed to wine critics loosely awarding 95-96 points on average-quality wines.

At the end of the day, there is too much BS around wine and wine critics. Complications exist to make industry professionals appear more relevant (a bit like the law). Most wines could be graded as follows:

0 - Undrinkable/off
1 - Barely drinkable/on last legs
2 - Average, generic
3 - Above average, good typicity
4 - Excellent
5 - World class / Reference standard

Most wines are 2-3. And the grading can be qualified with whether a wine is at that quality now, or where it is expected to reach.

[rens]

0 - Undrinkable/off
1 - Barely drinkable/on last legs
2 - Average, generic
3 - Above average, good typicity
4 - Excellent
5 - World class / Reference standard

I think this system was pioneered (and patented) by Ian Tickle

[gpk]

0 - Undrinkable/off
1 - Barely drinkable/on last legs
2 - Average, generic
3 - Above average, good typicity
4 - Excellent
5 - World class / Reference standard

I think this system was pioneered (and patented) by Ian Tickle

A vote for the above, simple and effective.

[vovo]

It would be good if someone could create a norm sample. Essentially a representative sample of wines available for sale transferred under laboratory conditions to unmarked containers and sent to wine critics. Call them norm 1 through 20 or whatever.

Thus under completely blind conditions a scale for a certain critic can be understood in comparison to others without any bias on the winemaker or winery.

For example reviewer x might rate norm 1 wine 96 points and reviewer y 93 points.

Hopefully with a sample of 20 and then subsequent years a comparative scale can be worked out by the discerning consumer.