Data Mining Donald

OK, here's a tidbit of news that not many of you will hear anywhere else. My friend and secret informant at MS, Donald Farmer, has just left the SSIS team in favor of the data mining team. Now, I talked to Donald about this last week, and he doesn't think he's blog-worthy, but that's where he's wrong and I'll tell you why. After talking to Donald about the reasons for his move, and what he hopes to accomplish, i

OK, here's a tidbit of news that not many of you will hear anywhere else. My friend and secret informant at MS, Donald Farmer, has just left the SSIS team in favor of the data mining team. Now, I talked to Donald about this last week, and he doesn't think he's blog-worthy, but that's where he's wrong and I'll tell you why. After talking to Donald about the reasons for his move, and what he hopes to accomplish, it was clear to me that this is something we all need. Data mining is the future, and as of yet, it's still far too complicated for the ordinary IT guy to grasp. Donald's hope is to bring a much simpler process to data mining, and make it so that any DBA or even educated business user can do it. Why do I think we all need Donald to make this leap? Well, because put simply… Donald actually cares about it. There's a fire in his voice that you don't hear often in this industry anymore, and anyone with that kind of fire can make things happen. It's like being a chef. Someone can teach you how to cook, but you'll never be truly great unless you're a real foodie. That's what Donald is I guess… a data mining foodie! I have another friend at MS right now who's trying to learn to code C++. The only problem is his heart's not in it and I can tell he'll learn just enough to get by. He'll never be great.

So for this reason, I think that Donald is very blog-worthy, and I'm personally very excited at the prospect of having someone with that level of excitement and commitment to data mining looking at the problems I face every day… ok, well, every other day anyway. It's funny isn't it... when somebody's excited about something it's contagious. They get you excited about it too, and I'm psyched about data mining right now.

But hey guys, don't take my word for it. Here are Donald's exact words to me about his move.


I can't see myself as blog-worthy, but certainly I can tell you what I'm up to and why.

My new role will be Principal Program Manager for the SQL Server Data Mining team. It's not really an up or down move. There's just so much fun to be had with customers and partners where the rubber hits the road. I asked for the move, and it's giving me a thrill to do this.

My motivation really goes back to why I am in the BI world at all. BI enables people to build and test their hypotheses against data of a volume and complexity that otherwise they would find difficult to grasp. Data integration is needed to do that, and so it was a great thing to work at that end of the process. But 6 years ago I actually joined MS to work on Analysis Services, because it was the analytic power that appealed to me.

I sometimes compare my passion for analysis with the amazement I felt when I was a kid, when I would help my dad in the garage, pumping the hydraulic jack. A 10-year old kid could lift a car off the ground with one hand! It was awesome. I feel the same about data analysis – imagine being able to make sense of every transaction of every customer for every year. That's a real buzz for me. Hypotheses can be formed, tested, rejected, applied, reviewed, shared ... all hopefully with relative ease.

Predictive analytics takes this a little further. Every single business uses predictive analytics - they just don't know it. When the village shopkeeper orders 2 boxes of valentine's cards, they are performing predictive analysis. In their case it is perhaps informed by knowledge of their customers, some analysis of past sales, and some gut feelings. Predictive analytics with data mining brings gut feeling to BI, captures it, models it and so on. I want to bring that to every user. I want small and medium businesses to have the capabilities of making data-driven decisions without needing to understand the complexities of algorithms or even how to model their data. That's going to be tough. And I want to get predictive analysis into situations where we’re not using it. I want to see DBAs receive predictive warnings about query performance degrading, or which databases are most likely to have issues under different loads – before it happens.

BTW, you'll see talking me more often about predictive analytics than about data mining. It's a fine distinction, but for me data mining is more about the techniques, while predictive analytics is more about the business cases and scenarios. I'm not going to be going all deep and academic on the algorithms – there is an awesome team of smart folks at MS who can do that. They will be marvelous to work with, I'm sure. For myself, I'll be concentrating on how we can change the landscape for BI users.

Finally, I just know I'm going to love working with the customers on these issues. Working with SSIS customers, we directly impacted their technical capabilities, and indirectly saw great benefits to their business. Working with predictive analytics customers, we'll get to see direct business benefits at first hand. That will be very satisfying.

OK Donald... all eyes are on you now brother... let's see whatchu got.

Related:

Copyright © 2007 IDG Communications, Inc.

How to choose a low-code development platform