Greetings from the field of data science, where the skill of drawing insightful conclusions from large datasets turns data into knowledge and stimulates creativity. It is not only a skill, but a requirement to grasp the foundations of data science in this era of unparalleled data abundance.
The book “Fundamentals of Data Science”, which you are currently holding, takes you on a thorough tour of the fundamental ideas that guide this dynamic and developing profession. This book, which was written with the intention of demystifying the complexity of data science, attempts to provide readers with the fundamental information and abilities required to successfully negotiate the complicated world of data.
We start by going over the fundamental ideas, highlighting the importance of computer science, statistics, and mathematics as the foundational fields of data science. These essential ideas provide the cornerstone around which the entire structure of data analysis and interpretation is built. After that, the book moves into the real world and walks readers through the complexities of data manipulation, exploration, and visualisation — skills that are crucial for anyone trying to make sense of the enormous amounts of data that are available to us.
As we go farther, the book presents the fascinating field of machine learning, elucidating its intricacies in a manner that is both practical and understandable. Case studies and real world examples show how machine learning algorithms can be used in a variety of disciplines to identify patterns, forecast results, and make well-informed judgements.
Contents –
Unit I DATA MINING
1. DATA MINING
1.1 Introduction – What is Data Mining?
1.2 The Scope of Data Mining
1.3 Tasks of Data Mining
1.4 Architecture of Data Mining
1.4.1 Data Source
1.4.2 Different Processes
1.4.3 Knowledge Base
1.4.4 Data Mining Engine
1.4.5 Pattern Evaluation Module
1.4.6 User Interface
1.5 Data Mining Process
1.6 Classification of Data Mining Systems
1.7 Major Issues in Data Mining
2. KNOWLEDGE DISCOVERY IN DATABASES
2.1 KDD vs Data Mining
2.2 What is KDD?
2.3 KDD Process Steps
2.4 What is Data Mining?
2.5 Why Do We Need Data Mining?
2.6 Why is Data Mining used in Business?
2.7 Why KDD and Data Mining?
2.8 Difference between KDD and Data Mining
3. DBMS vs DATA MINING
3.1 DBMS
3.2 Data Mining
3.3 What is the Difference between DBMS and Data Mining?
4. DATA MINING TECHNIQUES
4.1 Association
4.2 Classification
4.3 Prediction
4.4 Clustering
4.5 Regression
4.6 Artificial Neural Network (ANN) Classifier Method
4.7 Outlier Detection
4.8 Genetic Algorithm
5. PROBLEMS OF DATAMINING
5.1 Issues in Data Mining
5.1.1 Mining Methodology Issues
5.1.2 Performance Issues
5.1.3 Diverse Data Types Issue
5.2 Challenges of Data Mining
6. DM APPLICATIONS
6.1 Scientific Analysis
6.2 Intrusion Detection
6.3 Business Transactions
6.4 Market Basket Analysis
6.5 Education
6.6 Research
6.7 Healthcare and Insurance
6.8 Transportation
6.9 Financial/Banking Sector
Unit II DATA WAREHOUSE
1. DATA WAREHOUSE
1.1 Introduction to Data Warehouse
1.2 Definition
1.3 Characteristics of Data Warehouse
1.3.1 Subject-oriented
1.3.2 Integrated
1.3.3 Time-variant
1.3.4 Non-volatile
1.4 History of Data Warehouse
1.5 Data Warehousing Objectives
1.6 Data Warehouse Requirement
1.7 Advantages of Data Warehouse
1.8 Foundational Pieces of a Data Warehouse or Component of Data Warehouse
1.9 Data Warehouse Models
1.9.1 Enterprise Warehouse
1.9.2 Data Mart
1.9.3 Virtual Warehouse
2. MULTIDIMENSIONAL DATA MODEL
2.1 Working on a Multidimensional Data Model
2.2 Features of Multidimensional Data Models
2.3 Advantages of Multidimensional Data Model
2.4 Disadvantages of Multidimensional Data Model
2.5 OLAP (Online Analytical Processing)
2.6 Types of OLAP
2.6.1 Relational OLAP (ROLAP)
2.6.2 Multidimensional OLAP (MOLAP)
2.6.3 Hybrid OLAP (HOLAP)
3. DATA CLEANING IN DATA MINING
3.1 Procedures for Data Cleaning
3.1.1 Remove Duplicate or Irrelevant Observations
3.1.2 Fix Structural Flaws
3.1.3 Remove Undesirable Anomalies
3.1.4 Handle Missing Data
3.1.5 Validate and QA
3.2 Techniques for Cleaning Data
3.2.1 Ignore the Tuples
3.2.2 Fill in the Missing Value
3.2.3 Binning Method
3.2.4 Regression
3.2.5 Clustering
3.3 Process of Data Cleaning
3.3.1 Error Monitoring
3.3.2 Make the Mining Process More Uniform
3.3.3 Check for Redundant Information
3.3.4 Data Research
3.3.5 Make the Mining Process More Uniform
3.3.6 Communicate with the Team
3.4 Usage of Data Cleaning in Data Mining
3.4.1 Data Integration
3.4.2 Data Migration
3.4.3 Data Transformation
3.4.4 Data Debugging in ETL Processes
3.5 Characteristics of Data Cleaning
3.5.1 Accuracy
3.5.2 Coherence
3.5.3 Validity
3.5.4 Uniformity
3.5.5 Data Verification
3.5.6 Clean Data Backflow
3.6 Tools for Data Cleaning in Data Mining
3.7 Benefits of Data Cleaning
4. DATA INTEGRATION AND DATA TRANSFORMATION
4.1 Data Integration
4.2 Why is the Data Integration Important?
4.3 Data Integration Approaches
4.3.1 Tight Coupling
4.3.2 Loose Coupling
4.4 Issues in Data Integration
4.4.1 Entity Identification Problem
4.4.2 Redundancy and Correlation Analysis
4.4.3 Tuple Duplication
4.4.4 Data Warfare Detection and Backbone
4.5 Data Integration Techniques
4.5.1 Manual Integration
4.5.2 Middleware Integration
4.5.3 Application-based Integration
4.5.4 Uniform Access Integration
4.6 Data Transformation in Data Mining
4.7 Data Transformation Techniques
4.7.1 Data Smoothing
4.7.2 Attribute Construction
4.7.3 Data Aggregation
4.7.4 Data Normalisation
4.7.5 Data Discretisation
4.7.6 Data Generalisation
4.8 Data Transformation Process
4.8.1 Data Discovery
4.8.2 Data Mapping
4.8.3 Data Extraction
4.8.4 Code Generation and Execution
4.8.5 Review
4.8.6 Sending
4.9 Advantages of Data Transformation
4.10 Disadvantages of Data Transformation
4.11 Ways of Data Transformation
4.12 Data Reduction
4.12.1 Methods of Data Reduction
Unit III MINING FREQUENT PATTERNS
1. MINING FREQUENT PATTERNS
1.1 Basic Concepts
1.1.1 Market Basket Analysis: A Motivating Example
1.1.2 Frequent Itemsets, Closed Itemsets, and Association Rules
1.2. Frequent Itemset Mining Methods
1.2.1 Apriori Algorithm: Finding Frequent Itemsets by Confined Candidate Generation
1.2.2 Generating Association Rules from Frequent Itemsets
1.2.3 Improving the Efficiency of Apriori
1.2.4 A Pattern-growth Approach for Mining Frequent Itemsets
1.3 Summary
Unit IV CLASSIFICATION
1. CLASSIFICATION
1.1 Basic Concepts
1.1.1 What is Classification?
1.1.2 General Approach to Classification
1.2 Decision Tree Induction
1.2.1 Decision Tree Induction
1.2.2 Attribute Selection Measures
1.2.3 Tree Pruning
1.2.4 Scalability and Decision Tree Induction
1.2.5 Visual Mining for Decision Tree Induction
1.3 Bayes Classification Methods
1.3.1 Bayes’ Theorem
1.3.2 Naive Bayesian Classification
1.4 Rule-based Classification
1.4.1 Using IF-THEN Rules for Classification
1.4.2 Rule Extraction from a Decision Tree
1.4.3 Rule Induction Using a Sequential Covering Algorithm
1.4.3.1 Rule Quality Measures
1.4.3.2 Rule Pruning
1.5 Lazy Learners (or Learning from Your Neighbors)
1.5.1 k-Nearest-neighbor Classifiers
1.5.2 Case-based Reasoning
1.6 Prediction
1.6.1 Accuracy
1.6.2 Precision and Recall
Unit V CLUSTER ANALYSIS
1. CLUSTER ANALYSIS
1.1 Applications
1.2 Typical Requirements of Clustering in Data Mining
1.3 Major Clustering Methods
1.3.1 Partitioning Methods
1.3.2 Hierarchical Methods
1.3.3 Density-based Methods
1.3.4 Grid-based Methods
1.3.5 Model-based Methods
1.4. Evaluation of Clustering
MODEL QUESTION PAPERS