天美传媒

Skip to content
Science & Technology

Millions of tweets are a gold mine for data mining

Computer scientist Henry Kautz likens Twitter to a kind of distributed sensor network. Hundreds of millions of tweets are posted to the platform each day, with each user observing and reporting on some aspect of the world.

Unlocking big data

 

A Newscenter series on how Rochester is using data science to change how we research, how we learn, and how we understand our world.

 

聽鈥淓ach report is very noisy,鈥� says the Robin and Tim Wentworth Director of the Goergen Institute for Data Science at the 天美传媒. 鈥淏ut the aggregate results can be reliable.鈥�

Those results can provide information to meet all kinds of challenges鈥揻rom public concerns regarding health, safety, and the environment, to private ones regarding client and customer satisfaction and changing consumer tastes.

Tracking sickness and disease

Kautz and his team have used Twitter to 听补苍诲 , enabling health officials to respond much more quickly to disease outbreaks, and even to forecast when and if a specific individual will fall ill.

The Las Vegas Health Department field tested the nEmesis app developed by Kautz and his team to .聽The researchers found that the tweet-based system led to citations for health violations in 15 percent of inspections, compared to 9 percent using the traditional random system. That resulted in an estimated 9,000 fewer food poisoning incidents and 557 fewer hospitalizations during the course of the study.

Increasing business transparency

Huaxia Rui is a big believer in transparency. That鈥檚 why the assistant professor at the Simon Business School uses data science to delve deeply into Twitter鈥搊ne of the most transparent of our social media鈥� to study the relationships between companies and their customers.

Rui studies how companies respond to tweets, with the goal of increasing transparency about customer satisfaction across multiple industries.

airplane taking off, with sample tweets shown all around

For example, working with Simon professor Abraham Seidmann and PhD student Priyanga Gunarathne, Rui analyzed more than 450,000 Twitter messages to and from three major airlines. The researchers found that all three airlines were . The study raises interesting questions about fairness, but also concedes that airlines 鈥渕ay have limited resources to handle all requests for engagement.鈥�

鈥淲hen you call an airline or any company to complain about it, only you and the company know about it,鈥� Rui says. 鈥淚f you鈥檙e unhappy, what can you do? File a lawsuit? Most people won鈥檛 do that.鈥�

Twitter postings, on the other hand, are instantly public. 鈥淚n general its a good idea to improve the sharing of this data, and increase the transparency, so that people can see in real time what companies are doing and whether their customers are happy,鈥� he says.

Allocating resources

Rui has also found ways in which Twitter data might help both businesses and consumers operate more efficiently.

In one of his first studies involving the social media platform, Rui and two fellow researchers analyzed the impact of four million tweets on box office sales for 63 movies. So-called 鈥渋ntention tweets鈥濃�揻rom people who hadn鈥檛 seen the movies, but indicated they wanted to鈥揳ppeared to have a greater effect on box office sales than 鈥減ositive tweets鈥� from people who had actually seen the movies. Why? Rui cites the dual effect of intention tweets: They are a clear indication that their authors intend to see a movie, and also make their followers aware of the movie, possibly influencing them to see it as well.

How might a savvy business use this kind of information? Imagine you鈥檙e the manager of a retail store and it鈥檚 two weeks before Black Friday. If you鈥檙e scanning Twitter, and detect a surge in 鈥渋ntention鈥� tweets showing an interest in one of your products, 鈥淭hat could be useful for determining your staffing and inventory,鈥� Rui notes.

Geotagged tweets could narrow such staffing and inventory decisions to single regions, even individual stores.

The benefit for consumers? They may be less likely to find long lines or empty shelves on Black Friday if their local stores have done their Twitter 鈥渉omework鈥� in advance.

Taking the pulse of the voters

Jiebo Luo, associate professor of computer science, PhD students Yu Wang, and their colleagues tracked the Twitter followers of Donald Trump, Hillary Clinton, Bernie Sanders and other candidates last year to better understand the dynamics of the 2016 presidential campaign.

鈥淲e wanted to understand how each of the candidate鈥檚 campaigns evolved, and be able to explain why someone won or lost,鈥� says Luo, an associate professor of computer science.

Though the researchers did not set out to predict who would win the recent presidential election, their exhaustive, 14-month study of each candidate鈥檚 Twitter followers鈥揺nabled by machine learning and other data science tools鈥搊ffers tantalizing clues as to why the race turned out the way it did.

In another study with potential political applications, Luo and his students used images extracted from Twitter to train computers to determine what sentiments are likely to be elicited by images.