“Fast Onboarding” Pt. 2 – a modern solution

In my previous blog post, I presented the challenges in extracting business value from big data by adapting to new rules and datasets in a process I like to call fast onboarding. So how can we design a system for fast onboarding? Technology infrastructure can play a role. Fast onboarding, with or without that name, has been the motivation for a number of technology innovations for decades. One can even argue that fast onboarding was the motivation a half century ago for moving system design from hardware (hard to change) to software (easier to change). In modern times, we see advanced technologies whose relevance comes entirely from their ability to support fast onboarding. NOSQL databases and business rule systems are two such ... [ Read More ]

“Fast Onboarding” Pt. 1 – a classic business challenge

In this two-part blog series, I’m going to first discuss various frustrations of data management professionals and then the solution they need to extract value from the massive volumes available to them. I want to start by telling three different stories of information challenges, spanning decades of information processing history. What do they have in common? A credit card company creates a system to detect patterns of card usage that indicate the possibility of fraud. But the fraudsters catch on, and change their tactics. So the bank catches on, and changes its fraud check policies. How can the fraud detection system keep up? A contractor provides information systems to a government office. New legislation is enacted regarding ... [ Read More ]

Facing the Challenges of Big Data

Big Data is all the rage, especially since it received a new shot in the arm by a recent Gartner report.  Everyone is interested in it, and Gartner [1] has thrown its hat into the ring for a definition of it.  But even so, many people use these words in different ways. What is Big Data, and why all the interest in this topic now? For many people, Big Data is synonymous with search optimization (for many people, all of data science is synonymous with search optimization).  For commercial websites, visitor behavior equals revenue.  Understanding their behavior is essential for growing a web-based business and staying competitive.  So we gather data about our visitors, their habits, and try to glean some understanding of what does and ... [ Read More ]