Dataconomy
  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
Subscribe
No Result
View All Result
Dataconomy
  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
Subscribe
No Result
View All Result
Dataconomy
No Result
View All Result

Democratizing Data Assets: Learning From Data, Big and Small

by Kirk Borne
May 15, 2014
in Machine Learning, Topics
Home Topics Data Science Machine Learning
Share on FacebookShare on TwitterShare on LinkedInShare on WhatsAppShare on e-mail

When we devote so much time and energy talking about Big Data, are we neglecting the important things that you can do with Small Data?

Maybe, but… probably not.

Looking beyond the Big Data hype helps us to capture real value from advanced analytics on data, big and small.

The drumbeat of Big Data dialogue in social media, in the press, and everywhere merely highlights the important roles that data and analytics are now playing in all sectors. While we read, think, and dream about Big Data, we also realize that the “big” in Big Data refers to more than just data volume. We know all about Big Data velocity, variety, value, and more. For example: independent of data volume, you can have lots of variety in your data, you can have very tight real-time (data velocity) constraints, and you can derive huge value from your data assets. So, I believe that the deluge of Big Data discussions are not actually diverting our attention from small data, but they are indeed democratizing data assets — causing us to give more attention to how we can learn from data, both big and small.

So, how do we “learn from data”?

In the field of Machine Learning, algorithms are usually categorized as Supervised, Semi-Supervised, or Unsupervised. The first and second of these usually require the use of historical training data to build and improve classification and predictive models — it is fair to say that (in most cases) the bigger the training data set, then the better (more complete, accurate, robust) will be our predictive analytics models.

The third category of Machine Learning (Unsupervised Learning) is essentially the purest form of Data Mining (in my opinion): it is data-driven, evidence-based, unfettered by models or preconceived notions regarding the patterns in the data. It is used to discover the patterns, anomalies, categories, correlations, and features in the data, both BIG and SMALL. This is true knowledge discovery from data. One article (Shabalin et al. 2009) described it this way: “unsupervised exploratory analysis plays an important role in the study of large, high-dimensional datasets that arise in a variety of applications”.  When performed with rigorous systematic scientific methodology (as opposed to random “fishing expeditions”), the data mining application of Unsupervised Machine Learning algorithms becomes “powerful Jedi” Data Science.

“Discovery from Data” and “Learning from Data” certainly become more effective when the data set is big, but most types of unsupervised learning work just fine with relatively small data. There are four broad categories of unsupervised learning that you can apply to your data (big and small). These are:

  1. Novelty Discovery (also known as outlier or anomaly detection, but I prefer to call this “Surprise Discovery” — finding the rare, unexpected, surprising thing in your data).
  2. Correlation Discovery (finding the patterns, relationships, and correlations in data — e.g., using principal component analysis, or independent component analysis, or the maximal information coefficient).
  3. Class Discovery (finding new classes of events or behaviors, including finding new subclasses of previously known classes, or discovering improved rules for distinguishing and disambiguating known classes and subclasses).
  4. Association Discovery (discovering connections between different things, people, or events; or finding unusual, improbable co-occurring combinations of attributes, things, or objects — this type of algorithm is usually at the heart of Recommender Engines — it is sometimes called Link Mining, or Market Basket Analysis, or Graph Mining — several fun examples of this are discussed in the final 7 minutes of the TedX talk “Big Data, Small World”).

Yes, of course, knowledge discovery from data (i.e., Learning From Data) is FUN, especially if it is unsupervised!


kirkKirk is a data scientist, top big data influencer and professor of astrophysics and computational science at George Mason University.He spent nearly 20 years supporting NASA projects, including NASA’s Hubble Space Telescope as Data Archive Project Scientist, NASA’s Astronomy Data Center, and NASA’s Space Science Data Operations Office. He has extensive experience in large scientific databases and information systems, including expertise in scientific data mining. He is currently working on the design and development of the proposed Large Synoptic Survey Telescope (LSST), for which he is contributing in the areas of science data management, informatics and statistical science research, galaxies research, and education and public outreach.


Interested in more content like this? Sign up to our newsletter, and you wont miss a thing!

[mc4wp_form]

 

Tags: Big Data InfluencersKirk BorneMachine LearningSmall DataWeekly Newsletter

Related Posts

Claim your Discord Nitro x Midjourney free trial now

Claim your Discord Nitro x Midjourney free trial now

November 28, 2023
Having troubles finding a solid design? Try Stockimg AI

Having troubles finding a solid design? Try Stockimg AI

November 28, 2023
Runway Motion Brush mesmerizes users, here is how it works

Runway Motion Brush mesmerizes users, here is how it works

November 28, 2023
Stable Video Diffusion (SVD) transforms static images to dynamic shorts

Stable Video Diffusion (SVD) transforms static images to dynamic shorts

November 28, 2023
Latest ChatGPT trends to give you viral and productive ideas

Latest ChatGPT trends to give you viral and productive ideas

November 28, 2023
Email meets AI—another way that Olympia is reshaping the future of work, says Olympia CEO Victoria Loskutova

Email meets AI—another way that Olympia is reshaping the future of work, says Olympia CEO Victoria Loskutova

November 28, 2023

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

LATEST ARTICLES

Claim your Discord Nitro x Midjourney free trial now

How to get Apple Music Replay 2023 and why you have to

Having troubles finding a solid design? Try Stockimg AI

Sports Illustrated AI-generated authors got caught

Runway Motion Brush mesmerizes users, here is how it works

Did you lose your Google Drive account too?

Dataconomy

COPYRIGHT © DATACONOMY MEDIA GMBH, ALL RIGHTS RESERVED.

  • About
  • Imprint
  • Contact
  • Legal & Privacy
  • Partnership
  • Writers wanted

Follow Us

  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
No Result
View All Result
Subscribe

This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy Policy.