How to approach a Social Media Analytics project?

In previous occasions we have dedicated blog posts to the Social Media analytics topic, however we still had not written about which are the most common questions our projects must answer or which are the objectives we should aim for when we plan to extract insights from a Social Media network.

What you should deliver

Last year, SAP gave a series of webinars focused on Social Media (that you can find here if you have a valid SAP user) where they shared some interesting ideas about what a social media project should deliver to a customer. As consultants, we put a great portion of our efforts in endorsing our projects with as much value as possible, which was one of the topics discussed in these webinars. From them I extracted 4 principles that we should try to align our projects with in order to accomplish our goal of delivering value from Social Media analytics.

• Speed: Social Media analytics must enable the customer to solve challenges faster through a comprehensive understanding of the technologies and experts related to their business. On the other hand, in a world where we have 6000+ tweets per second in average (source: https://twitter.com/TPS_watcher) there is a lot of information to analyze and not enough time to digest all of it, so we have to make sure that whatever our customer is analyzing from their social networks we are able to do faster!
• Impact: The data that you extract and produce needs to give the necessary information to the customer to enable them to drive higher quality outcomes and build a reputation among their targets in the social networks. If you know what your target is hungry for, the easier it gets to satisfy their needs. Your analytics must be capable of making the customer understand how much they stand out and to monitor their presence in the social networks.

• Insight: This principle is also related with the above in the sense that when your customers are constantly listening to what experts in their industry are talking about, they could benefit from that collective wisdom to innovate and differentiate themselves from other competitors. It is not just a matter of gathering data, it is also necessary to gather knowledge and trends. It also comes without saying that this also applies to monitoring what your customers' prospects are saying, an accurate and timely understanding of their needs or opinions, increases your customer’s probabilities of delivering what they want.

• Efficiency: With all the information that you deliver from the Social Media analytics, you can make it possible for your customer to reduce time and investments spent in the monitoring of their Social Media activity. You should deliver with a server in minutes (why not in real-time?) what several brains take to process in many hours of work.

What you should be asking

One of the main risks of these projects is that it is very easy to lose yourself in the gigantic amount of information that you can extract, so from the very beginning you have to determine the main questions that your customer wants answered and solely focus on them. In my opinion, the most common key points that you should always look for are:

• Who are the business influencers and what are they talking about?
• What is the competition doing and how are they using their Social Networks? How big is their impact compared to your customer's?
• Which are the Social Media networks that are best suited for your customer's needs?  Are they being used enough?

What are the benefits for your customer

Now, imagine yourself in a position where you have to sell the key benefits to a potential customer. He does not want to hear the whole “Social Media is the future” speech, he just wants you to get to the point… If your project can deliver these points below, you will be indubitably in good shape:

• Discover key business influencers and reach them
• Design effective strategies assessing the competition’s activities
• Understand if the message sent to the target has good visibility

If you have any questions or anything to add to help improve this post, please feel free to leave your comments. You may also find other interesting articles related to this topic in our blog here.

BI and Social Media – A Powerful Combination (Part 3: Sentiment Analysis)

In previous posts we have covered the role thatGoogle Analytics and Facebookplay in BI projects focused on Social Media Analysis. Therefore, it was only a matter of time before we covered Twitter - the most popular micro-blogging network you can find on the web. Besides that, it will not be rare to find analysts and reviews that consider Twitter the social network that can potentially deliver the highest amount of meaningful information to analyze.At this point, I guess everyone has a general idea of what Twitter is and what it delivers, so the objective of this article will be to make an overview of a Sentiment Analysis showcase that we built extracting data from Twitter with SAP BusinessObjects Tools. Then, in future articles we will cover each phase of the development in more detail. Generally speaking, we consider Sentiment Analysis as the process of identifying, extracting and measuring data from a subjective information source, such as customer surveys, opinion polls, or tweets as in our case.

Data Extraction

As in any BI project, the first step is to define the data that you need, and how to get it. Using SAP BusinessObjects tools, the best way to do this is to develop an Adapter for Data Integrator using the SDK that this tool includes in its installation folders (check this article from SAP SDN that proved to be very helpful).

However, to do the demo as quickly as possible, we used another approach:

• We developed a Java program that made use of Twitter’s getSearch API to extract tweets and place them in text files Note that for demo purposes this is more than enough, but for a broader project the flat files are not a satisfactory solution.

• With Data Integrator, we configured an ETL flow to extract the data from the files and store them in database tables to accumulate enough tweets to make the demo meaningful.

Also consider that in this phase it is very important to get comfortable with Twitter’s API and the different parameters that it uses so you can take advantage of it as much as possible.

Data Parsing and Sentiment Analysis

Once we were able to place the tweets in text files and customize the extraction parameters as we desire, then we could actually analyse the tweets to start delivering insight from them. To do so, we followed these steps:

• Get the raw tweets that we stored in the database before and perform a parsing process with Data Integrator, to get rid of the JSON format that Twitter API uses, enabling us to manipulate the tweets as text strings.

• Use the feature of Text Analysis that Data Integrator includes to perform the “Sentiment Analysis” process and classify the tweets in one of the different sentiment categories that we used. For the demo purposes that we had there is a SAP Blueprint called Text Data Processing Data Quality that contains Data Integrator jobs with a Voice of Customer implementation that already contains a set of extraction rules implemented for the English language. Therefore, you can make use of this blueprint and its rules to develop the Sentiment Analysis phase.

• Build a universe on top of the tables with the analyzed data in order to make it available for reporting with any of the SAP BusinessObjects tools that take an universe as data source, e.g. Xcelsius, WebIntelligence, Explorer, etc. In this step, we also made a use of an universe that came included in the same Text Data Processing Data Quality blueprint that we used for the point above.

Data Visualization

Finally comes the eye-catching part: present all the hard work you have done. To show the users how flexible this solution can be, we decided to present the data with Explorer and some Exploration Views built on top of its Information Spaces. However, as said before, if you build an universe on top of the tables that resulted from the Text Analysis process then you will have a great number of possibilities and tools to play with, in order to bring forth the presentation you want according to your requirements and objectives.

In future articles, we will cover each one of these sections in further detail. However, with this general layout we hope you get a good idea of what you need to do to make your Sentiment Analysis demo happen!

If you have any questions or anything to add to help improve this post, please feel free to leave your comments.

BI and Social Media – A Powerful Combination (Part 2: Facebook)

To continue with my Social Media series  (read the previous blog here BI and Social Media – A Powerful Combination Part 1: Google Analytics), today I would like to talk about the biggest social network of them all: Facebook. In this blog post, I will explain different alternatives I have recently researched to extract and use information from Facebook to perform social media analytics with SAP BusinessObjects’ report and dashboard tools. In terms of the amount of useful information we can extract to perform analytics, I personally think that Twitter can be as good or even better than Facebook, however, it has around 400 million less users. Facebook still stands as the social network with the most users around the world - 901million at this moment - making it a mandatory reference in terms of social media analytics.

Facebook

Before we start talking about technical details, the first thing you should understand is that Facebook is strongly focused on user experience, entertainment applications, content sharing, among others. Therefore, the user activity is more dispersed and variable as opposed to the real-time orderly fashion that Twitter gives us, which is so useful when building trends and chronological analysis. Hence, be sure of what you are looking for, stay focused on your key indicators and make sure you are searching for something that is significant and measurable.

Relevant Facebook APIs for Analytical purposes

The APIs (Application Programming Interface) that Facebook provides are largely directed at the development of applications for social networking and user entertainment. However, there are several APIs that can provide relevant information to establish Key Indicators that can later be used to run reports. As Facebook’s developer page1 states: “ We feel the best API solutions will be holistic cross API solutions.” Among the API’s that you will find most useful (labeled by Facebook as Marketing APIs), I can highlight the Graph API,  the Pages API, the Ads API and the Insights API. In any case, I encourage you to take a look at Facebook pages and guides for developers, it will be worth your time:

• Marketing Developer Program

• Marketing Developer Resources (with mentions of the APIs above)

• Facebook Marketing Solutions: http://www.facebook.com/marketing

Facebook APIs

Third party applications to extract data from Facebook

I only found a few third party applications to extract data from Facebook’s API that were comprehensive enough to ensure reliable access to data. Below are some alternatives designed for this requirement:

GA Data Grabber: This application has a module for the Facebook APIs, which costs 500USD a year. As in the case of Google Analytics, it has key benefits such as ease-of-use and flexibility to make queries. It may also be integrated with some tools from SAP BusinessObjects such as WebIntelligence, Data Integrator or Xcelsius dashboards through LiveOffice.2

Custom Application Development: It is the most popular option, as I already mentioned in my previous post about Google Analytics. The Facebook APIs admit access from common programming languages, allowing to record the results of the queries in text files that can be loaded into a database or incorporated directly into various tools of SAP BusinessObjects.

Implementation of a Web Spider: If the information requirements are more focused on the user’s interactions with your client’s Facebook webpage or any of its related Facebook applications, this method may provide complementary information to that which is available in the APIs. The information obtained by the web spider can be stored in files or database for further integration with SAP BusinessObjects tools. Typically, web spiders are developed in a common programming language, although there are some cases where you can buy an application developed by third parties, as the case of Mozenda.3

Final Thought

As I have mentioned in my previous post, in the area of Social Media new applications and trends are appearing at a hectic pace, a lot of changes are expected to happen, so it is just a matter of time until we have more and better options available. I encourage you to stay curious on Social Media analytics and its most popular networks, because right now this is a growing gold mine of information.

If you have any questions or anything to add to help improve this post, please feel free to leave your comments.

BI and Social Media - A Powerful Combination (Part 1: Google Analytics)

If you take a look at the latest Business Intelligence (BI) trends you will see a great deal of mentions revolving around the “Social Media” topic. There are a lot of interesting ideas out there and it seems to be certain that this will become even more prominent in the near future since it continues growing and it is becoming an intrinsic part of our society. This also means that the information related to Social Media becomes more valuable with each passing day, becoming the new “gold mine” for Business Intelligence consultants when handled correctly. No wonder there are a lot of us that want to start digging in!

In this blog post, I will share the research I recently made with the objective to determine the possible alternatives to extract and use information from Google Analytics to develop SAP BusinessObjects reports and dashboards. An important note is that I did not consider SAP BusinessObjects Data Integrator’s Unstructured Data and Text Analysis capabilities on purpose, in order to see what other options were out there.

Extracting data from Google Analytics

The first thing you must know is that in December 2011, Google released the Google Analytics Core Reporting API (in replacement of its old Data Export API) which allows us to extract data from its application. Although it is not rare to see the old API still being used, if you are starting a fresh development and have the opportunity to choose, then go with the new version of course. Considering these interfaces, I found four different strategies that stood above the rest.

Programmed Google Docs spreadsheet: It is a Google Docs spreadsheet available free of charge, which contains embedded Google Apps Script code to connect to the Google Analytics Data Export API. Once authenticated, it allows the user to define the dimensions, metrics and filters filling in the fields of the document with the desired values. Subsequently, this spreadsheet can be exported as a .CSV file and use it as a data source to be reported with several tools of SAP BusinessObjects such as Web Intelligence or Data Integrator, for example. Credits go to Mikael Thuneberg who developed this. 1

GA Data Grabber: It offers a 7-day trial version, the cost to purchase the Google  Analytics module is 299USD per year (at the present date). It consists of an Excel file containing Macros and Visual Basic code to run queries with any of the dimensions, metrics or filters available in the Google Analytics Core Reporting API. This is one of the best options I found so far, since it has a user interface that is very easy to use, good flexibility to select metrics, dimensions and filters, besides having a fairly comprehensive structure that facilitates the understanding of the type of information that can be obtained from the API. By having the data available in an Excel file there are more SAP BusinessObjects tools that can be involved in addition to Web Intelligence and Data Integrator, such as LiveOffice that allows direct integration with Xcelsius dashboards. 2

Reporting Utility of Google Analytics: This functionality is already included in the Google Analytics application. It is a relatively manual process, which requires the user to have some prior knowledge of the dimensions and metrics that can be obtained. However, custom reports can be created with specific information that can be exported to .CSV files so they can be incorporated into the reporting capabilities of SAP BusinessObjects.

Custom Application Development: If third party tools are not the desired option, then the best strategy to use is to develop a custom application in any common programming language, such as PHP, Ruby, Python, Java and JavaScript. The Google Analytics Core Reporting API lets you connect with these type of applications. In spite of representing the option that requires more effort to be implemented, it is also the one that permits to have a perfectly tailored, automatable and free solution. The data can be potentially stored in text files or custom databases from where any SAP BusinessObjects tools can draw information.

Official Documentation always does the trick

I did not want to get into too much technical detail because they are very susceptible to change through time, and it is always best practice to look at the official documentation published by Google. In this sense, I will leave you with a few links that will be very useful to deepen the technical knowledge necessary to develop an integrated solution with Google Analytics.

• Google Analytics Core Reporting API Overview:

https://developers.google.com/analytics/devguides/reporting/core/v3/

• Dimensions & Metrics Reference (to understand what information can be retrieved):

https://developers.google.com/analytics/devguides/reporting/core/dimsmets

• Google Analytics Core Reporting & Data Export API Change Log:

https://developers.google.com/analytics/community/export_changelog

• Google Analytics Core Reporting API Client Libraries:

https://developers.google.com/analytics/devguides/reporting/core/v3/gdataLibraries

• Google APIs Console:

https://code.google.com/apis/console

Final Thought

As with all things that become “the latest trend” and  develop at a hectic pace, a lot of changes are expected to happen, meaning that sooner rather than later there will be more and better options available. So my final thought is to keep an inquisitive eye for all that is moving around Social Media and its most popular networks, because this is the future that is already happening.

If you have any questions or anything to add to help improve this post, please feel free to leave your comments.

References

1 For more info: http://www.automateanalytics.com/2010/04/google-analytics-data-to-google-docs.html

2 For more info: http://www.gadatagrabbertool.com/