Big Data & Digital Marketing

Data is certainly growing at an exponential rate, and the Internet Of Things is just making the process stronger. However, the buzz has created some false myths that need to be busted.

Five biggest myths about big Data:

1. The Entire Data is Accessible

2. Entire Data is Needed

3. Offers Certainty

4. Big Data isn't Here to Stay

5. Granular Data is Better

Lots of information about Big Data from New Jersey Institute of Technology.

The upper half of the infographics introduces Big Data with the usual buzz words. The remaining has a lot of interesting statistics.

My 6 takeaways from the infographics:

1. $300 billion, what US could save in Healthcare

2. +60% in operating margin (retail)

3. 73% of companies have already increased revenues thanks to big data

4. 56% of IT decision makers believe that finding the right staff is the biggest challenge

5. "Query and reporting" is the Top 1 capability currently available

6. "Transactions" and "Log Data" are the most common source of data currently collected and analysed (88% and 73% respectively).

When a successful Data Science project is well presented, it usually looks very simple. It reminds me of some of the proofs in Calculus or Physics that I studied when an undergrad.

In fact, they just *look* simple, and they do so because someone has done an incredibly hard and difficult job before hand.

In Data Science projects, the hidden job is usually related to data: looking for data, cleaning the data, joining the data, realising you are missing some data and iterate.

Here are 6 possible trends in Big Data in 2015. I mainly the following 3:

1. Big Data will go mainstream -

so much that we might start dropping the term "Big" and just talk about "Data"

2. Everything with go on the cloud -

this will simplify the usage, cloud is simple, cheap and, above all, flexible

3. Analyses will become faster -

thanks to improvements in big data tools and technologies

In the past, emerging technologies might have taken years to mature. Now people iterate and drive solutions in a matter of months, or weeks.

The article presents the top emerging technologies and trends that should be on your watch list. Here are my best 4 pick:

1. Big Data analytics in the cloud

2. SQL on Hadoop: Faster, better

3. More, better NoSQL

4. Deep learning

Big Data can be extremely dangerous without a Big Brain to analyse them properly.

Huge data sets ALWAYS contain some relations: some of them are right (causation), others are simply wrong. It pertains to data analysis to uncover the truth.

"As we move into an era in which personal devices are seen as proxies for public needs, we run the risk that already existing inequities will be further entrenched.

[...]

This goes beyond merely conducting focus groups to confirm what you already want to see in a big data set. It means complementing data sources with rigorous qualitative research."

At the beginning there was Gartner, who defined Big Data with 3 Vs:

1. Volume

2. Velocity

3. Variety

Then the issue was raised that Big Data new insights can be actually quite uncertain, and so the 4th V arrived:

4. Veracity

But all this was still leaving out the deep core, the only reason why one wants to use Big Data:

5. Value

Data Lakes as the Big Data alternative to Data Warehouses.

Their main features are:

1. Contain ALL of the data, both structured and unstructured, internal and external;

2. Store all data in RAW format;

3. No SCHEMA is imposed on the data.

I like Data Lakes because they:

1. Are flexible

2. Break silos

This increases the success rate of Big Data projects.

Another way to look at this is:

Since the market offers fewer data scientists than needed, new tools are being developed so that less experienced professionals can analyse data in a productive way.

Will this "Data Discovery" be an evolution of self-service BI that we see emerging today?

I think that Microsoft products such as Power BI, or Excel Power Query and Power Pivot, are worth mentioning in this context.

Sometimes it's good to go back to the basics:

What is Big Data?

Gartner in 2011 gave the 3V definitions, today we have a better understanding ad we find more appropriate to add two more concepts.

1. Volume: how much data?

2. Velocity: how data grows or moves?

3. Variety: about the shape

---

4. Veracity: about the reliability

5. Value: about the ROI

The first 4 points describe what Big Data is, the fifth one reminds us that a Big Data project is relevant if it adds value.

I repeat it every time I can: always state your goal *before* embracing a Big Data project.

What is the biggest problem you have? Why do you want to collect all this data? What kind of insight are you looking for? Just saying 'insight' and 'innovation' is a wonderful thing, but first and foremost you need to focus.

And one more thing: a successful Big Data project is not a matter of having a super-hero data scientist, but a talented TEAM.

Interview to IEEE Fellow Michael I. Jordan, Pehong Chen Distinguished Professor at the University of California, Berkeley.

1. Deep Learning has nothing to do with Neuroscience

People continue to infer that something involving neuroscience is behind deep learning, and that deep learning is taking advantage of an understanding of how the brain processes information, learns, makes decisions, or copes with large amounts of data. And that is just patently false.

For issues of higher cognition—how we perceive, how we remember, how we act—we have no idea how neurons are storing information, how they are computing, what the rules are, what the algorithms are, what the representations are, and the like.

So we are not yet in an era in which we can be using an understanding of the brain to guide us in the construction of intelligent systems.

3. A Big Data Winter is waiting ahead

When you have large amounts of data many of your inferences are likely to be false. It’s like having billions of monkeys typing. One of them will write Shakespeare.

A lot of people are building things hoping that they work, and sometimes they will. And in some sense, there’s nothing wrong with that; it’s exploratory. But society as a whole can’t tolerate that. Eventually, we have to give real guarantees. Civil engineers eventually learned to build bridges that were guaranteed to stand up.

So with big data, it will take decades, I suspect, to get a real engineering approach, so that you can say with some assurance that you are giving out reasonable answers and are quantifying the likelihood of errors.

If nothing changes, there will be a “big-data winter.” After a bubble, when people invested and a lot of companies overpromised without providing serious analysis, it will bust. And soon, in a two- to five-year span, people will say, “The whole big-data thing came and went. It died. It was wrong.”

A large number of Big Data charts try to represent the following progression:

1. What Happened? (descriptive analytics), 

2. Why Did It Happen? (correlation analytics), 

3. What Will Happen Next? (predictive analytics), and 

4. What Should I Do About It? (prescriptive analytics).

Introducing an organization to BI is a learning process from the start.

Many enterprises see where they want their Business Intelligence platform to be, without having a baseline understanding of the roadmap to achieve that end product.

Everybody talks about Big Data, but few people give references.

This article is a useful source of several *Reliable* statistics about Big Data:

1. Data explosion (Forbes)

2. Obama, the Big Data president (Washington Post)

3. Information management (Economist)

4. Aadhar: Big Data with 600M individuals, in India (Forbes)

5. Big Data Market (Gartner)