Today, we can store and process so much data that we have nearly captured reality; no more sampling biases/ errors or related issues – this is my definition of Big Data; not tera or peta bytes! If you have measured the entire population (or close to it) and not sample just a small fraction, resulting data is BIG Data!
Analytics, Machine Learning, Data Science, pattern recognition, statistical modeling, data mining, knowledge discovery, predictive analytics, data science, adaptive systems, self-organizing systems, . . . There ARE subtle technical differences but let us just call it “Analytics”, at least in business applications!
When you hear “dynamics”, time always comes to mind first but it is only one of the many possibilities. Dynamics could be over any independent variable! Context-sensitivity to better identify a word within a sentence is also an example of dynamics.
As a card-carrying Neuroscientist, I can say that creating Artificial Intelligence by mimicking the human brain seems to be a fool’s errand. If you think about neuronal axons along which electrical spikes travel, they are like conducting wires with insulation scraped off every few millimeters! THEN, the whole jumble of these billions of wires are dunked in a salt solution! Try sending your TCP/IP packets along such a network . . . I believe that the electric fields that these leaky wires produce have a major role to play in how the brain works. However, in silicon implementations of artificial brain today, electric fields are actively suppressed by careful physical layout and ground planes!
Now, there is a glimmer of hope for Artificial Intelligence of this sort. Our ancestors saw flying birds and dreamt of flying; our planes don’t flap their wings but they fly really, really well! Similarly, there may be a Wright Brothers who finds that last twist in the story and creates Artificial Intelligence that works like human brain but NOT by “flapping the wings” . . .
Instead of AI, I am a believer in IA or “Intelligence Augmentation”! When Doug Englebart wrote to ARPA about IA in the 1960’s, he could not have foreseen what Big Data can do. Englebart’s “Mother of All Demos” was really about Communication augmentation – computer mouse, video conferencing, hypertext, etc. With Big Data & Analytics, NOW we can truly do Intelligence augmentation! This is why all of us Data Scientists ought to be excited to be working in this field.
There is hardly ever a business solution that is “one and done”! What do I mean by that? One-shot solution that will, FOREVER, solve the problem . . . like life-long immunity from chickenpox (even that is not perfect – shingles can show up later in life). Almost all business problems I have seen are more like needing yearly “flu shots” – monitor the outcomes of the first solution, tweak the mix, administer again after a while and so on. Continuous Intelligence Augmentation . . .
After much customer interactions, I am convinced of the following simple “syllogism” in Analytics:
Businesses need actionable insights from data – “PRESCRIPTIVE” Analytics.
Prescriptive Analytics business solutions are like “flu shots” . . . treat, monitor, update & treat again over time.
Businesses need closed-loop Analytics at the right time-intervals!
That is Analytics as a system or “SYSTEMS Analytics”.
As Data Scientists, we have a duty to educate business clients to subscribe to this view, what I call, “goal-seeking” or tracking solution concept. The first solution may be just 80% but track and improve over time. With such realistic expectations, your customer will be delighted if the trajectory is good and fast. For a data science company, this is good – recurring revenue! Clearly, a win-win situation. So, collect data over time . . . and provide “Goal-seeking and Tracking solutions” within a Systems framework.
Syzen Analytics, Inc., has developed what is known as SYSTEMS Analytics as an effective Machine Learning tracking solution framework to address the dynamics of Retail Commerce. Details are explained in a Youtube video: “Future of Analytics – a roadmap”. https://youtu.be/1TAYLQw3u9s
As a guidance for your IA implementations for Retail or other business problems where “dynamics” (tracking solution) is important –
- We have sketched out a roadmap of increasingly complex tools that can be brought to bear on Analytics or Machine Learning of today.
- These tools provide high-value features in terms of system parameters, a framework for closed-loop real-time Analytics and ways to possibly accommodate the networked nature of data sources.
- Theories of all the techniques discussed in the video are fully or partially developed but will require additional development to reach their full potential for Analytics applications.
- Breakthrough business applications of the later milestones will require significantly more development in collaboration with business domain experts.
A prerequisite for performance at a high level in business is the ability to understand and manage complexity. Complex systems to be managed properly requires a ton of data at the right time. BIG Data provide us the data we need; to put these data to work in order to take us to the high levels of complexity required while still managing it, we have to anticipate what is about to happen and react when it happens in a closed loop manner. Predictive Analytics will allow us to push our “system” to the edge (without “falling over”) in a managed fashion. This is why businesses embrace Predictive Analytics – to manage businesses at a high level of performance at the edge of complexity overload.
You may ask, “Why not address e-commerce with Analytics?!” Look at 2014 data for US – Total In-store Retail: $4 TRILLION; e-commerce: $300 Billion or 6.4%; you want to address the MUCH bigger business opportunity! E-commerce growth is levelling off and projected e-commerce plus m-commerce total in 2018 is expected to approach 11%. (Source: eMarketer, 2014). By then, OMNI channel movement will make the distinctions among in-store, e-commerce and m-commerce irrelevant – sales attribution to any single entity will not make sense anymore!
Whenever there are constraints such as display space, warehousing cost, advertising dollars, personalization challenge, etc., there is a “product density” problem. Analytics can be put to work (“relevance” engine, for example) to optimize the solution!
Remember “Goal-seeking and Tracking solutions” as a System (see #6). A complete characterization of the essential elements of retail dynamics is captured in this canonical diagram. Business objective of ANY retail commerce is to increase customer acquisition and retention; then all good business results follow. Business owners have three levers to affect change – Marketing, Loyalty and Merchandising.
As businesses push to higher levels of performance (see #8), higher fidelity models are going to be necessary to produce more accurate and hence valuable predictions and recommendations for business operations.
ALL data are spatio-temporal! At the simplest to more complex levels –
- Data can be considered isolated at the simplest level – a “snap shot”.
- Then we realize that data exist in a network with mutual interactions.
- In reality, data exist in *embedded* forms in “influence” networks of one type or the other which are distributed in time and space – a “video”!
Spatial extent of data (distance) can be folded into time if we assume a certain information diffusion speed. Graph-theoretic methods do not account for time dimension. For accurate analysis, no escaping Dynamics over Time; meaning the use of differential (or difference) equations . . . and Systems Theory!
