ABC's missing stories

When we started this blog a little while ago we cast an eye on the productivity of ABC's news service by looking at the number of news stories ABC posted to its online News Archive. Our productivity survey pointed to a massive decline in the productivity of ABC News. The ABC responded by capping the number of stories posted daily to 250. It retrospectively applied this to its online archive effectively wiping easy access to 10,000s stories from 2003 onwards. This effectively undermined our capacity to compile the information. Sadly its also means that past stories are no longer available on line. A glimpse of the number of stories ABC removed can be had by using the "Wayback Machine"and looking through the internet archive. For Friday August 2007, ABC removed around 100 stories!

Screen shot showing ABC's news archive for August 24, 2007 showing 10 pages of storied (25 stories per page)

Screen shot from the internet archive showing ABC previously posted 14 pages. What happened to the missing stories?

We requested further information at the time but were never provided with a satisfactory response. We have requested this be looked into again.

How petty is ABC NEWS that it would spite the nation by removing an important source of historical information rather than permit an assessment of its declining productivity be measured.

Culling the Archive part 2- Dear Mr Scott.

Dear Mr Scott,

REGARDING ABC ONLINE NEWS ARCHIVE-10000s of stories culled...why?

I made an enquiry regarding the manner in which ABC manages its online news archive a little while ago to your senior staff including Alan Sunderland, Bruce Belsham and Angela Clark (see HERE). None of these staff members has been at all helpful in providing any information about the manner in which stories are posted to the archive, or about recent changes that have resulted in the apparent removal of 10,000s reports from the archive since 2003.

I have been using the Online News Archive as a measure of ABC news productivity. Since 2007 there has been a marked decline in the number of stories posted the archive (see HERE and HERE). 

I am trying to establish if this is a result of a policy decision related to managing the archive, or a true reflection of declining news coverage. Perhaps reflecting a fall in the number of regional stories being covered by the ABC at the expense of opinon pieces?

As I mentioned to your staff above, ABC's NEWS archive represents an important, growing source of historical news information, and as such some explanation as to why the number of posts to it have been falling is deserving of some explanation. If this is a matter of policy, then perhaps it can be corrected so that all the news makes it into the archive, and not a "cherry picked" sub-sample. ABC's growing archive will no doubt be important in helping future historians unpick past events, the more complete it is the better it will be.

I note that this year ABC is consistently posting 250 stories for Tuesday-Friday (ed actually its the whole week M-F), including numerous opinion pieces not previously included. Is the inclusion of opinion an attempt to cover for falling coverage of news stories? 

Of greater concern I also note that the remainder of the archive, since 2003, appears to have been retrospectively culled of reports such that the total number of stories posted each week day has also been reduced to 250. This has involved removal of 10000s of reports and seems to represent a deliberate destruction of the value of the archive for historical research purposes.

I have collected records of the number of stories posted to the archive for each day in the month of June from 2003 to present as part of the productivity report. The numbers can be confirmed through the online internet archive (see HERE). I have not had the opportunity to go back through each year. For June 2003 the effect is dramatic with over 590 stories removed. For June 2004 it seems there are 171 less now compared to 2010 when we first used the archive to report on ABC's productivity. For the year of 2008 when we originally counted a record 8894 stories for June, ABC's archive now contains 6618 stories. It seems that for just one month ABC have culled 2276 reports from history! And this just for the month of June, extended over the whole archive it means 10,000s of stories have been evaporated, akin to literally cutting the stories from the pages of an archived newspaper

Specific questions:
1. What is ABC's policy, if any, regarding its Online News Archive?
2. Do the falling number of items since 2007 reflect a fall in news productivity, or are they the consequence of a change in records policy?
3. On what grounds does ABC decide to include news items in its archive?

4. Why did ABC cull its news archive to remove easy accessibility to 10000s of stories? Which stories? Why? How-what criteria were used for the cull?

5. Can we expect to see the full archive restored? When?

The lack of a satisfactory timely response and deliberate refusal to cooperate with seemingly legitimate requests on this matter is of some concern hence I have cc'd this email to Senator Conroy, Malcolm Turnbull, Senator Fierravanti-Wells, along with a representative from the mainstream press to draw attention to the issue. 

Regards

Marc Hendrickx

ABC NEWS WATCH

Culling the Archive

Some screen shots below showing the effect of the cull of reports from ABC's News Archive:
 The Wayback Machine, the Internet archive, has preserved some glimpses of ABC's Archive for posterity. We looked at June 16, 2008 when ABC News posted a record 392 reports. The Wayback machine archive captured on 2 February 2011 confirms our earlier count, showing 15 pages with 25 reports each, with the 16th page containing 17 reports. As of this morning the ABC's achive shows just 10 pages, each with 25 reports for a total of 250 reports, meaning access to 142 reports through the archive have been removed, effectively culled (see comparison of screen shots below-click to enlarge). It seems that reports prior to about midday, in this case, have been sent down the memory hole. 

The missing stories such as this one (Skiers take advantage of alpine snow) still appear to be available if you know the URL.

Why the ABC have placed a limit on the number of stories in its archive remains a mystery. It seems at this point that attempts to thwart our productivity survey at least one possible reason, incompetence always another.

2012 Productivity Report: ABC hiding the decline-Part 2 the scandal

Perhaps using George Orwell's 1984 as an instruction manual ABC have apparently, sometime in the last 12 months, removed large numbers (10,000's)  of stories from its online news archive. Is this a means to hide the dramatic decline in its news productivity? Which stories were removed and why? What a radical re-writing of history! Or is it just a technical glitch?

ABC NEWS WATCH has been looking at the productivity of ABC news since 2010. As a measure of ABC's news productivity we have been using the number of stories posted to ABC's online news archive in June and its staff numbers from annual reports to help gauge its output. ABC senior staff have been reluctant to assist us with information about how the archive is managed (see correspondence in part 1). The figures show a dramatic decline in productivity since 2007.

Looking over the number of stories posted for this year it became apparent that ABC have set a limit of stories that are posted to its archive in June this year. The limit appears to be set at 250 stories per day Monday to Friday. Weekend figures do not appear to have been adjusted.
To our surprise it appears this limit has been retrospectively applied to ABC's entire online archive since 2003. If true, this means that literally 1000s of stories have been culled (UPDATE make that 10,000s).  
 I have records of the number of stories posted for each day in the month of June from 2003 to present. I have not had the opportunity to go back through each year. For June 2003 the effect is dramatic with over 590 stories removed. For June 2004 it seems there are 171 less now compared to 2010 when we first reported on ABC's productivity. For the year of 2008 when we originally counted a record 8894 stories for June, ABC's archive now contains 6618 stories. It seems ABC have culled 2276 reports from history! And this just for the month of June, extended over the whole year it means 10,000s of stories have been evaporated.


Which stories? Why? How-what criteria were used for the cull? Or is this just a fat finger moment?
More on this as we find out what has happened.


We have archived some of the revised archive pages at webcite for posterity (eg HERE).