Twitter data yields valuable public health trends

By filtering through 2 billion public tweets, John Hopkins University scientists discovered health patterns, including misuse of medications

Twitter has once again proven itself to be more than a mere diversionary tool for chirping about the latest Justin Bieber sighting or sharing all manner of photos with the world (intentionally or otherwise). Computer scientists at John Hopkins University have discovered the service can be used as a valuable tool for tracking public health trends, according to the institution.

Mark Dredze and Michael J. Paul used specially groomed software to sift through 2 billion public tweets posted between May 2009 and October 2010. From all that big-data noise, the scientists gleaned useful information and trends pertaining to health matters, such as allergies, flu cases, insomnia, cancer, obesity, depression, and pain. They even discovered findings previously unknown to members of the medical community.

The first challenge in the project was to develop filters capable of differentiating between tweets about actual health issues and tweets containing health-related words or phrases, such as "fever" (such as "zomg i've got bieber fever!!!") or "gas" (e.g. "Gas prices are way too high!")

The system's filters enabled the duo to reduce the pool of 2 billion tweets to around 200,000 relevant ones, from which Dredze and Paul were able to track certain health trends by time and also place, thanks to user-provided geographic info. Discoveries included determining when allergy and flu seasons had peaked in various parts of the country. "We were able to see from the tweets that the allergy season started earlier in the warmer states and later in the Midwest and the Northeast," Dredze said.

Beyond pinpointing various health ailments via Twitter posts, the researchers were able to record many of the medications that ill tweeters consumed via posts such as, "Had to pop a Benadryl ... allergies are the worst," according to the university.

Some tweets revealed that users were misusing medication, thus exposing "serious medical misperceptions," according to Paul. For example, some people tweeted that they were taking antibiotics for flu when, in fact, antibiotics don't work on the illness, as it's a virus. "This practice could contribute to the growing antibiotic resistance problems," Paul said.

The researchers acknowledged limitations to the sort of information that can be extracted from tweets, noting, for example, that Twitter users tend to be younger, so data about senior-citizen health isn't well represented. Additionally, they found that users tended not to comment on symptoms more than once, thus making it difficult to determine types of ailments and how long they lasted.

This story, "Twitter data yields valuable public health trends," was originally published at Get the first word on what the important tech news really means with the InfoWorld Tech Watch blog. For the latest developments in business technology news, follow on Twitter.

Copyright © 2011 IDG Communications, Inc.

How to choose a low-code development platform