Want to dig into Big Data from Stackexchange websites?

Aleksandra Puchta-Górska

14 Nov 2012.2 minutes read

Krzysiek Grajek’s new project – SECharts enables you to see all the data aggregated and presented nicely with colourful charts.

The site aggregates data from 37 sites using publicly available Stack Exchange Data Dump from August 2012 ranging from sites about bicycles to those for database administrators.

Want to dig into Big Data from Stackexchange websites?

So what might this all data be useful for?

Check popularity of different Stackexchange websites

Let’s start with your bet? Which of the websites do you think is the most popular measured in questions per month created? Of course Stackoverflow is the most popular – it takes 83% of all the questions ever asked on SE, but which websites are second and third? And here comes the answer with just one glance on the chart – in August 2012 that second most popular was askubuntu, and it goes head in head with the websites about math.

You can also see how the popularity of the 10 most popular sites has been changing since 2009 on the welcome screen.

Want to dig into Big Data from Stackexchange websites?

Find experts in the field

One of the great features is that you can check top users of every site in terms of votes and reputation. It’s a great way not only to find experts in the field, but also dedicated users of particular technologies.

Check trends within every field

Ever wondered what terms are the most popular in UX design? You can check that easily by choosing tags category. In 2010 that was ‘forms’, in 2011 ‘website design’ and this year ‘usability’ seems to be the most popular term.

Want to dig into Big Data from Stackexchange websites?

Check how long will you wait for an answer

The overview of every website gives you information on the average waiting time for every question and number of views per question. And there are nice charts showing how many percent of questions were actually answered.

Want to dig into Big Data from Stackexchange websites?

Have fun while reading the most popular questions

Ok, this is the best part (at least for me). I checked the most popular questions for every website and they were really fun. Like on the website ‘Seasoned advice’ it is ‘How can I chop onions without crying?’ or ‘Is it possible to cook a whole fish in a dishwasher?’ And who wouldn’t like to know the answer…

Want to dig into Big Data from Stackexchange websites?

Or another interesting: ‘Why do people clear the screen multiple times when using a calculator?’ – this one is from UX design websites and you can check the answer here: http://stackexchagecharts.s3-website-us-east-1.amazonaws.com/ux.html

And there are many, many more of them. Apart from fun, especially software comapnies might monitor the frequent questions and react to some flaws or simply lack of information.

Wonder why did Krzysiek start this project?

This is how he answers: ‘At first, I wanted to see who has the most points at Stackoverflow and which questions are most often ‘liked’ and ‘up-voted’. Then I started learning Pig script, so averages, top’s, sorting and simply wanted to get something interesting from this huge amount of data.’

As you can see our team members are REALLY creative and who knows what other Big Data projects are being baked just right now…

If you want to learn more, here is a story by Krzysiek on his blog: http://www.softwarepassion.com/big-data-analysis-with-hadoop-pig-and-stack-exchange-data-dump/

Check the SECharts by yourself: http://stackexchagecharts.s3-website-us-east-1.amazonaws.com/index.html

And you are more than welcome to post comments below on how to make use of the website .

Blog Comments powered by Disqus.