Skip to content

News

Published on

Since 1999, Magnum Opus has been a leading data mining tool making association discovery better and faster for everyone.

BigML is the best platform for Machine Learning on the internet.

G.I. Webb & Associates are excited to be partnering with BigML to provide the best tools and environment for association discovery.

As a result, G.I. Webb & Associates are no longer offering new Magnum Opus  licenses or downloads. We will continue supporting our licensees as usual.

BigML.com

 

Published on

SDM-awardWe are delighted to receive the SDM15 Best Research Paper Honorable Mention award.

The Society for Industrial and Applied Math (SIAM) International Conference on Data Mining (SDM15) Awards Committee selected 4 papers for awards from nearly 400 submissions.

View the presentation here.

And here is a link to the paper and its bibliographic details:

    [URL] Petitjean, F., & Webb, G. I. (2015). Scaling log-linear analysis to datasets with thousands of variables. Proceedings of the 2015 SIAM International Conference on Data Mining, pp. 469-477.
    [Bibtex] [Abstract]  → Related papers and software 

    @InProceedings{PetitjeanWebb15,
author = {Petitjean, F. and Webb, G. I.},
booktitle = {Proceedings of the 2015 {SIAM} International Conference on Data Mining},
title = {Scaling log-linear analysis to datasets with thousands of variables},
year = {2015},
pages = {469-477},
abstract = {Association discovery is a fundamental data mining task. The primary statistical approach to association discovery between variables is log-linear analysis. Classical approaches to log-linear analysis do not scale beyond about ten variables. We have recently shown that, if we ensure that the graph supporting the log-linear model is chordal, log-linear analysis can be applied to datasets with hundreds of variables without sacrificing the statistical soundness [21]. However, further scalability remained limited, because state-of-the-art techniques have to examine every edge at every step of the search. This paper makes the following contributions: 1) we prove that only a very small subset of edges has to be considered at each step of the search; 2) we demonstrate how to efficiently find this subset of edges and 3) we show how to efficiently keep track of the best edges to be subsequently added to the initial model. Our experiments, carried out on real datasets with up to 2000 variables, show that our contributions make it possible to gain about 4 orders of magnitude, making log-linear analysis of datasets with thousands of variables possible in seconds instead of days.},
comment = {Best Research Paper Honorable Mention Award},
keywords = {Association Rule Discovery and statistically sound discovery and scalable graphical models and Learning from large datasets and DP140100087, efficient ml},
related = {scalable-graphical-modeling},
url = {http://epubs.siam.org/doi/pdf/10.1137/1.9781611974010.53},
}
    ABSTRACT Association discovery is a fundamental data mining task. The primary statistical approach to association discovery between variables is log-linear analysis. Classical approaches to log-linear analysis do not scale beyond about ten variables. We have recently shown that, if we ensure that the graph supporting the log-linear model is chordal, log-linear analysis can be applied to datasets with hundreds of variables without sacrificing the statistical soundness [21]. However, further scalability remained limited, because state-of-the-art techniques have to examine every edge at every step of the search. This paper makes the following contributions: 1) we prove that only a very small subset of edges has to be considered at each step of the search; 2) we demonstrate how to efficiently find this subset of edges and 3) we show how to efficiently keep track of the best edges to be subsequently added to the initial model. Our experiments, carried out on real datasets with up to 2000 variables, show that our contributions make it possible to gain about 4 orders of magnitude, making log-linear analysis of datasets with thousands of variables possible in seconds instead of days.

Published on

View our panel on Video Lectures:


A Data Scientist’s Guide to Making Money from Start-ups
Geoff Webb, Foster Provost, Ron Bekkerman, Oren Etzioni, Usama Fayyad, Claudia Perlich

We also wrote a paper based on the panel discussion:

    [URL] Provost, F., Webb, G. I., Bekkerman, R., Etzioni, O., Fayyad, U., & Perlich, C. (2014). A Data Scientist's Guide to Start-Ups. Big Data, 2(3), 117-128.
    [Bibtex] [Abstract] 

    @Article{ProvostEtAl14,
Title = {A Data Scientist's Guide to Start-Ups},
Author = {Provost, F. and Webb, G. I. and Bekkerman, R. and Etzioni, O. and Fayyad, U. and Perlich, C.},
Journal = {Big Data},
Year = {2014},
Number = {3},
Pages = {117-128},
Volume = {2},
Abstract = {In August 2013, we held a panel discussion at the KDD 2013 conference in Chicago on the subject of data science, data scientists, and start-ups. KDD is the premier conference on data science research and practice. The panel discussed the pros and cons for top-notch data scientists of the hot data science start-up scene. In this article, we first present background on our panelists. Our four panelists have unquestionable pedigrees in data science and substantial experience with start-ups from multiple perspectives (founders, employees, chief scientists, venture capitalists). For the casual reader, we next present a brief summary of the experts' opinions on eight of the issues the panel discussed. The rest of the article presents a lightly edited transcription of the entire panel discussion.},
Keywords = {Big Data},
Url = {http://dx.doi.org/10.1089/big.2014.0031}
}
    ABSTRACT In August 2013, we held a panel discussion at the KDD 2013 conference in Chicago on the subject of data science, data scientists, and start-ups. KDD is the premier conference on data science research and practice. The panel discussed the pros and cons for top-notch data scientists of the hot data science start-up scene. In this article, we first present background on our panelists. Our four panelists have unquestionable pedigrees in data science and substantial experience with start-ups from multiple perspectives (founders, employees, chief scientists, venture capitalists). For the casual reader, we next present a brief summary of the experts' opinions on eight of the issues the panel discussed. The rest of the article presents a lightly edited transcription of the entire panel discussion.