Don’t stop – there is a lot more to discover out there!

There is a rumour out on the street that says that the age of the Zettabyte is near. Now that is a realistic number when measuring internet traffic. But could there possibly be analytical environments of this size? There are lots of Teradata customers that are in the Terabyte range, and they do unique stuff with their data. Is there any point in having huge data volumes available in itself? A Zettabyte equals 1024 Exabytes, which equals 1024 Petabytes – and the Petabyte club is still an exclusive circle of Teradata customers (although their volumes are increasing at a breath-taking pace.) What is their interest in “big data”? Couldn’t there be a point when large data volumes become an obstacle, rather than an opportunity for analytical purposes?

Listening to the big guys, you don’t get the impression that they worry about size. Large data volumes rather seem to be their playing field on which they try out new tricks. While many data warehouse architects are heavily concerned with economics, these guys are worried that they do not capture enough detail. Oliver Ratzesberger of eBay specifically quoted one of his colleagues who said that you should never throw away any data at all because you never know what you may want them for in a few years’ time. The knack is to have the data ready when the business idea pops up rather than having to wait an extra 12 or 30 months until you have collected enough material to test it on.

This appears to be the attitude that sets the Petabyte club apart from (and often ahead of) other players: they are moving away from using data to confirm suspected answers towards discovering genuinely new things. It requires them to capture high quality detailed data as opposed to summary data. For example, storing information about transactions will enable you to identify some long-term patterns. But it is the interactions with the company that a customer has before and during the purchase that tell the whole story, especially when the sale does not take place in the end (which is clearly the more interesting case.) On websites, simple usability details may turn out to be deal-breakers en masse. But you will never find them unless you track and analyze customer interactions. The club members make sure they can do so when they notice that they need to.

The lesson to learn from these giants is that when you are laying out your data warehouse, it makes sense to look beyond the first few years when you do standard reporting and a bit of analysis. At a later stage, when you have learned to make full use of your insights and when your organization has begun to direct more and more questions at the analytics department, you will want to predict, react to and trigger business events. For example, look at the telecommunications sector. A few years ago, some people questioned the benefits of storing call detail records. The reality today is that if you cannot analyze these data by now, you’ll probably find it hard to stay in business at all, given your competitors’ analytical advantages. The same is happening with OSS data now. This data is turning out to be extremely valuable if you want to understand customer experiences with your services. Are there, for example, any bandwidth bottlenecks in any area that actively drive customers away? And isn’t there any way you can spot these problems in real-time and rectify them right away? Well, there will be, unless you have decided against capturing those data on a detailed level in the first place.

It’s worth to keep this in mind as social media data, and all those other new data sources, are about to be integrated into enterprise analytics. An assortment of data won’t keep you at the head of the field. I am aware that this looks like a massive challenge to many enterprises. If handling the Petabyte was like breaking the sonic wall, handling the Zettabyte appears to be like trying to travel at speed of light. Well, I’d say just don’t stop (as Freddy Mercury has put it), keep on pushing – and we will get there all together! The consensus at the panel was that in the next 3-5 years, we will see the required innovations. Now ain’t that worth keeping the faith?

Socialization at the Partners

Darryl McDonald has elaborated his concept of socialization of data (socializing is a key thing at conferences, so why have we never thought of those data before) today, and if you want to watch him doing so you can see for yourself here. People have expressed their curiosity about this concept in response to my post yesterday, so I will try and depict it here, in the way I understand it.

Darryl basically argued that if you want to draw value from your data, you need to do three things: integrate, explore and share them. Now this is what you do in databases of any size, no matter whether you are running a mere data mart or a full-scale enterprise data warehouse. In my own simple words, Darryl’s message was that we are used to building our own little model worlds – some smaller, some bigger – and tend to be content with them for some time, while at the same time data out there is virtually exploding. The examples he gave were social media, mobile devices and sensors. All of these new data sources hold huge business potentials once enterprises start to tap into them, which is already happening, and will be even more of it in two years’ time or so. What Darryl was really telling us was that we should think up ways to find and grasp those new opportunities. Teradata is providing the technical means for it.

What it all comes down to is what I call the “realization gap”, which occurs because we have been preached in the past years that (textbook style) innovation is led by business: we need it, so we invent it. But I think that at the moment, changes in business will be driven by new technologies, not the other way round. Just remember that mobile devices and geospatial data came first and all those location-based services that we appreciate now followed. In fact, only 18 months ago Teradata CTO Stephen Brobst predicted, not reported, these services. This gap has been closing with frantic speed.

Back to the technical side, the three steps of integrating-exploring-sharing your data. There was a challenge for us that we have risen to. On the one hand, we know that deeper insights require integration of new data with existing data – for example, making customers suitable offers requires more than just their location data, obviously. On the other hand, there is a whole new diversity both of data types and analytical tools specializing on these types. This is why Teradata has put so much effort on integration with partner tools and why we are looking for new ways in which our platforms can work together. From in-database analytics, data labs and the unified LDM to the alignment with Hadoop systems, it all serves to support the vision of an enterprise ecosystem that is open for data and people, as Darryl put it.

So we at Teradata are enabling tomorrow’s business breakthroughs because we have a clear idea which data are being generated and because we are making them available for analysis. Darryl’s platform socialization of data is our bit to help close the realization gap and at the same time take the pulse of all those pioneers that brace themselves to take those new opportunities. After all, ideas get born when people with different backgrounds engage in discussion, don’t they?

So much for now. If you want some lighter inspiration, what views you can get from truly granular material, have a look at the Sand Artists who performed at today’s general session.

Time-travelling with your data warehouse

This headline is not an allusion to the inevitable jet lag that many visitors from our region still feel. It’s about the most spectacular announcement that has been made today at the Teradata Partners conference today, in my view at least. And, as I indicated yesterday, there was a whole wealth of technologically interesting news at today’s media briefing. More on those later.

Think about this: Have you ever wondered why on earth you have taken a decision that turned out to be awfully wrong? If this has ever happened to you, I am sure that you had at least one well-worked out theory. You will say that you didn’t pay enough attention to X or that you were too much preoccupied with Z and, most importantly, you vow to never make this mistake, any mistake again. Data warehousing is all about decision support. So if you want to know why decisions were made in a certain way, they should give a good idea, shouldn’t they? You should be able to simply rewind those data and look at the full view of the time.

This was actually not so easy for various reasons. Data change over time, some of the data simply gets overwritten, and this is why we used to take snapshots of these data every now and again. But this is not enough to capture when exactly certain changes occurred and what effect this had on dependent data. This used to require a lot of “manual” work (complex coding, really) that is now being done by the Teradata Database 13.10. The effect will be much like with geospatial in the recent past: everyone relied on zip codes because geographical representation was such a hustle – and suddenly every mobile application uses it. So it bet that the temporal rewind button will be everywhere in the near future. And you can use it to find out what you really did know when you took a certain decision. You can even use it to demonstrate this to third parties if you want to. And there are lots of business questions that you can answer a great deal more easily and accurately with this feature. I find this whole idea really fascinating, maybe because I remember the movie Back to the Future so well. It was released in Italy exactly 25 years before Teradata’s temporal data announcement. If I could rewind my life to the 80s, I would probably be having a good time listening to an U2 vinyl. I wonder whether I would feel more like “Marty McFly” (Michael J. Fox) who doesn’t really grasp what is happening to him or like “Doc” who feels completely in control all the time.

- Teradata offers a Unified Logical Data Model now that allows enterprises to map all of their value chains from beginning to end (and possibly crossing boundaries of industry that were previously catered for with distinctive LDMs)

- More performance, storage and usability with the platform family, which has a full five members by now

- Teradata Relationship Manager has incorporated modules to engage customers on smartphones, iPads and social media sites within seconds – with all the knowledge stored in the data warehouse at hand

- Teradata Accelerated Analytics Portfolio includes lots of new data types, emerging technologies, partner optimizations, and tools for application development

And there is also some news from our partnerships that I won’t be able to cover today. As I have said, there was a lot. And hey look, I can’t go massive-parallel. I’m a mere human being – and I plan to keep it that way for the foreseeable future. But I pledge that I’ll come back to you as soon as I can. See you.

Another Euphoric State Conference

I have arrived in San Diego at this year’s Teradata Partners Conference now, and as usual when I reach the West Coast of the United States I feel somewhere in between a terrible jet lag (the difference to Milano and most of Europe is nine hours) and feverish excitement that tells me to just shrug off the fatigue and enjoy. When I flew in on Saturday, I found myself humming “it never rains in Southern California” to the San Diego rain and some people even seemed to nod at me in silent agreement. Now, who was it again that referred to California as “Euphoric State?”

The only real calamity that plagues this piece of land (apart from the occasional earthquake) seems to be wildfires that break out from time to time. In fact, back in 2007 a wildfire raged in the area near Rancho Bernardo where both the Teradata Research and Development and the Global Support Center (GSC) reside. The authorities wouldn’t run any risk and ordered an evacuation that forced Teradata to close its facilities temporarily. They were not harmed, and a few days after the fire (and the smoke) was gone, operations resumed as usual. In the meantime, the GSC was facing a major challenge, having to ensure that Teradata’s 24/7 customer support would be kept up. It turned out that its business continuity plan worked splendidly and operations progressed despite of the fires. For example, when a retailer reported performance challenges at one of its nodes, Teradata Customer Services were on cue and solved the problem as speedily as usual. The GSC was rising to the challenge alright.

The proximity to Teradata Research and Development at this year’s conference makes sense because – I think I can reveal that much – that there will be a lot of technologically relevant announcements from Teradata and its partners on Monday. During the next few days, I plan to follow around some journalists the Expo area and report back to you which partners they are most interested in and what questions they debate. The same goes for the various conference highlights, most importantly the general session on Monday where our Chief Marketing Officer Darryl McDonald will explain his concept of the socialization of data.

So much for now. Let the show begin!

A busy final day in Berlin

After three intensive days, this conference is coming to an end now. Apart from the pleasure of meeting so many friendly faces who share an interest in our discipline, I am enjoying the feeling that I have once again gathered a multitude of fresh ideas – or should I say, loose ends – that I will take home and reflect upon. And I hope you feel the same way. Here are just a few thoughts on today’s keynote and workshops.

Stephen’s bid for the BICC
Stephen Brobst, Teradata’s CTO, made a strong case for the concept of the BI competence centre (BICC) that would serve as a link between IT and business users. The idea is to say good-bye to the project-based implementation approach and form permanent teams with experts from both sides. “Good plans are not set in stone. They embrace change, and don’t prevent it,” he said. The BICC would spread analytical skills and best practice examples throughout the enterprise, and quickly identify new business information needs and find their solutions. This entity would define standards and the overall BI architecture – but more importantly, it would adjust the BI environment to business users’ preferences and requirements, not vice versa.

Healthcare workshop
There was also the healthcare workshop that we set up together with SAS. Jürgen Kellermann and Matthias Kleinschmidt of GKV Spitzenverband, a German association of statutary health insurance based here in Berlin, gave us some insight into the good work both they and their members do. Apparently, they do not only have to make extra sure that they comply with privacy regulations – there is also the competition among its members that they have to take into account. Still, they manage to establish meaningful benchmarks and produce figures that serve as a valuable basis in negotiations with the medical associations. In other words, even in a highly sensitive environment, there is leeway to examine anonymous health data for opportunities to cut costs and increase medical effectiveness. Which is good news considering that practically all healthcare systems (in the Western hemisphere, at least) face similar problems.

Automotive podcast
As for our special automotive track at this conference, I am sorry that I couldn’t find the time to cover it. From what I hear, it has been a very productive encounter of two (up to now) distinctive cultures, engineering and IT. On the one hand, data analysis has already been a common method among engineers, for example in quality assurance. On the other hand, the enterprise-wide approach that Teradata adheres to means: First, that there are many other valuable data warehousing applications for the automotive industry, for example in supply chain management or CRM. Secondly, this leads to more available data, such as reported early warranty cases, opening up new analytical opportunities in quality management. As for the latter, this podcast with Duncan Ross explains the potentials.

Remember the Marauder’s Map?
Finally, just one more remark. I remember that last year, during the Istanbul conference, I was formulating the vision of some kind of Marauders Map on our smart phones that maps the marvels of the city we’re in adjusted to our individual preferences and social networks. Judging from what I have seen on the smart phones of our participants, we have already come halfway here. There is no shortage of apps that help you to find your way through a city you don’t know. As for the analytical part which is necessary to further customize this information, I am sure that we will see a lot more in the future.
For now, let me thank you for your participation. I hope you found this conference as inspiring as I do. I wish you all a safe and comfortable journey home.

Facing and embracing a changing world – with sneakers and energy

If I had to sum up what I’ve learned at yesterday’s Teradata Enterprise Intelligence Summit, it would probably go like this: the world is changing and that’s OK, as long as we respond accordingly. Geoff Burch claimed that managers need to notice when their staff are doing things right – one of the basic rules of motivation. But how do you empower people to get things right in the first place? Hermann Wimmer’s answer during the closing session: By handing them the information they need. He made an impassionate plea: “Set information in your organization free so that every decision is a smart decision”.

The eagerly awaited keynote of Joschka Fischer, the former Vice-Chancellor and Foreign Minister of Germany, was equally passionate. I love the story of Joschka Fischer wearing sneakers at his inauguration as the first minister of the German green party in 1985, which my German colleagues told me. Pretty bold in a surrounding that couldn’t be any more formal! In addition to making a fashion statement, wearing sneakers also demonstrates a dynamic and forward-looking mindset. It seems to me that Fischer started a trend with it. Or do you still find it odd that Steve Jobs always gives his famous presentations wearing sneakers.

Apart from his forward-looking “sneakers-attitude”, Fischer is also known for being a strong advocate of environmental protection. That doesn’t mean he longs for pre-industrialized times, he rather embraces high tech as part of the solution. You could have heard a pin drop when Fischer drew an impressive picture of the ongoing shift in power from the Western countries to the emerging new economic powerhouses in Asia, Latin America and Eastern Europe. At last, billions of people are reaching a decent standard of living through progress in society and technology – and that is definitely a good thing, a dream coming true! There are downsides to this development however. They include shortages in energy and water supply as well as worldwide environmental pollution and climate change – quite severe consequences. Fischer’s conclusion: Since we only have one world, the challenges of this century can only be managed if all mankind act as one, which so far has never been the case. Waste of resources, especially energy, must come to an end quickly, because we are definitely running out of time. Instead, we urgently need to make energy supply more efficient and to enforce the adoption of renewables.

We at Teradata share this view. And we strongly believe that data warehousing is indispensible here. The “smart grid”, for example, would not work without a precise and dynamic, real-time picture of the electricity networks. The same applies if you want to give your customers an opportunity to find out about the carbon footprint for, let’s say, flying from Milan to Berlin, and to choose the “greenest” option accordingly. In addition to saving energy and enabling ecologically responsible decisions, data warehousing also makes energy supply more reliable. After all it is, in a way, a long supply chain, in which you need to forecast demand and order pre-products timely and accurately, for example. This is definitely one field in which advanced analytics will play a crucial role in the next ten years.

Faraway, so close

Before I dart off to my first of our Enterprise Intelligence Summit, let me share some impressions from day one. Having ploughed through a packed agenda, we let the evening wind down at Tempelhof Airport, which was closehttp://www.blogger.com/img/blank.gifd in 2008 and has since earned a reputation as an event location. It has been the setting of a number of films like Billy Wilders “A foreign affair”, or Wim Wenders’ ingenious “Faraway, so close” with the famous U2-Song “Stay”. More importantly, it was the centre of the Berlin airlift in 1948/9 when railway and road access to the Western parts of the city were blocked. It was Tempelhof where the so-called “Raisin Bombers” landed and supplied the people of Berlin with food and other essentials for nearly eleven months.

How times have changed! Groceries and consumer goods are abundant – and retailers have to compete for the customer by creating the best shopping experience. Which basically means offering a consistent customer dialogue across sophisticated sales channels, as Dr. Gerd Wolfram, Head of CIO-office, METRO Group, explained. Metro’s future store initiative is following the Zeitgeist by offering customers a mobile shopping assistant (MSA) that creates shopping lists and provides additional product information. As for the relevant data, no one would seriously complain that they have become generally scarce recently; the real question is how to leverage the data as a strategic asset for the organization. With data at your disposal you’re as close as you can be to understanding every aspect and process of your business. But if you’re not able to unearth the treasure, meaning to draw the information from the data – you’re far away from it, maybe even further than if the data didn’t exist at all…Faraway, so close.

Certainly Professor Hans Rosling would agree with that. He impressively demonstrated that decisions based on presumptions are no better than those made by chimpanzees. There is no alternative to considering all the available data out there if you want to understand what is happening out there. Forget about gut decisions.

Trend-spotter Markus Lindkvist is in line with Rosling. He claims that our thinking can be at odds with reality. Do you still feel like twenty, although you’re already past your fifties? A common attitude, according to Lindkvist. This is because we all tend to see things not as they are, instead we see them as we are and call that being focused. Future is about unpredictability and things we’ve never thought about, says Lindkvist. And I would like to add: you’d better take care that you are really aware of what’s going on right now, because it will be challenging your status quo.

Before I’ve got to go, let me quickly call your attention to some of my personal highlights of today, which are the general sessions with Bob Pritchard, Joschka Fischer and Geoff Burch. I definitely won’t miss those.