Hacked off by statistics

Leave a comment
fuckwittery / journalism

Here’s an interesting story from the Press Association, today:

One in five university students have hacked into computer systems, from using someone else’s online profile to breaching internet shopping accounts, a survey has found.

The idea that “One in five university students have hacked into computer systems” is pretty remarkable. Of course there’s no indication given of what the survey means by “hack”, but even assuming it’s something vague, along the lines of “bypassed some sort of computer security measure to use something without someone’s consent” it’s still a very high percentage.

So what does the research itself say? You can’t look it up online, but as a journalist I’m lucky – I can ask the PR for a copy. So I did.

“Tried” and error

The most immediate problem is that the survey doesn’t actually ask the question “have you ever hacked into a computer”. It does ask “Have you ever tried hacking”, and 23% of those surveyed answered “yes”.

Having “tried hacking” and “hacking into” are not the same thing. If I walked to any computer in this office and attempted to log in using the password “hellokitty” I could say, perhaps, that I’ve “tried hacking”. I would not have hacked into anything. Next paragraph of the story:

The survey of 1,000 university students found 37% had hacked Facebook profiles, 26% targeted emails and 10% breached online shopping accounts.

This suggests that 37% of the 1000 students surveyed (there were, in fact, 1001, but never mind) have hacked into Facebook. That’s completely wrong. In fact, the question is “have you or a friend ever hacked into the following”, and it was only put to those who answered “yes” to having “tried hacking”.

So, 37% of the 23%, or 10.5% of the total sample, have either “hacked into” Facebook, or rather said-they-thought-they-knew-someone-who-has. You have to wonder if they’re even correct about that. The same error applies to the 26% (really 7.4%) and 10% (3.8%) figures. Next:

Nearly half of the students (46%) had also had their own social networking or email accounts hacked, with 41% saying their passwords to university networks had been abused by a third party.

Again, this is just bollocks. The 46% figure refers to the question “Have you or any of your friends had their (sic) Social Networking/email account hacked?”. The 41% refers to the question “has anyone ever abused your passwords” with no mention of university networks.

The article goes on. And it’s not alone: the same mistakes – particularly attributing percentages to the whole group, when in fact they refer to the much smaller subset, and ignoring the “or a friend clauses” in the questions – can be found in other stories too.

So where does this all come from? Well, here’s the start of the press release:

Research published today by IT security experts, Tufin Technologies, reveals that 23% of college and university students have hacked into IT systems. Of these hackers, 40% waited until after their 18th birthday before their first hacking attempt. On a positive note, 84% of 18-21 year olds recognised that hacking is wrong. However, 32% identified that hacking is ‘cool’ and worryingly, for the targets of hackers in this age group, 28% considered hacking to be easy.

This research, which was supported by the Association of Chief Police Officers (ACPO) – builds on a study carried out in March amongst teenagers. The teenage research survey revealed similar attitudes towards hacking, although only 18% considered hacking to be easy, suggesting that hackers’ experience develops through their teenage years. Both surveys found that there was no gender bias in hackers with an equal split between boys and girls.

The survey which was carried out amongst 1000 College and University students from across 5 London Universities and 3 Northern Universities showed that just over one in three students said that they hacked for fun. A further 22% cited curiousity as their main reason for hacking. An entrepreneurial 15% revealed that they hacked to make money. This was further reflected in the types of sites that had fallen victim to these youngsters. The survey found that 37% had hacked facebook accounts, 26% email accounts with 10% breaching online shopping accounts. Although 39% of hackers use their own computer, others have used public computers and networks with 32% a university machine and 23% using an internet café.

Unfortunately, the study also discovered that nearly half of the students (46%) had fallen foul of hackers having had either their social networking or email accounts breached. A further 41% said that they had had their passwords to university networks abused by a third party.

You can read the whole thing here.

It’s easy to see how the errors were copied-and-pasted out: 23% “have hacked”, and “the survey found that 37% had hacked facebook accounts, 26% email accounts with 10% breaching online shopping accounts”, with no mention of the “or a friend” clauses. These errors don’t take much research to spot, but it seems that nobody checked the survey findings, let alone how the survey was conducted.

Call the cops!

Incidentally, you might wonder why this survey, which is clearly designed to promote the computer security firm behind it, mentions the Association of Chief Police Officers, and can be found on that organisation’s website.

After all, ACPO doesn’t exist to help firms promote their services through annoying press releases – it’s partially funded by a grant from the Government. And yet it appears in the release, and pops up in many of the news pieces:

The research by Tufin Technologies and the Association of Chief Police Officers (ACPO) found that…

(Source here)

The report, commissioned by Tufin Technologies and the Association of Chief Police Officers in the U.K

(Source here)

So I asked the ACPO press office, which is easy to do if you’re a journalist but not so much if you just happen to have read one of the stories. The answer? “We weren’t involved in the research at all.”

Splendid.

Incidentally, I also asked the National Union of Students what it made of the whole thing. Its response, in its entirety, was:

“We can’t comment on a survey without full details of the methodology – how was the sample selected, what’s the full demographic of the group, etc.”

How terribly sensible. Clever people, students.

* Sample: me. Completely made up margin of error: 2%.

Leave a Reply