Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 35 hours
Course Outline
Day 01
Overview of Big Data Business Intelligence for Criminal Intelligence Analysis
- Law Enforcement Case Studies: Predictive Policing
- Big Data adoption rates in Law Enforcement Agencies and strategies for aligning future operations around Big Data Predictive Analytics
- Emerging technology solutions including gunshot sensors, surveillance video, and social media
- Leveraging Big Data technology to mitigate information overload
- Interfacing Big Data with Legacy data systems
- Foundational understanding of enabling technologies in predictive analytics
- Data Integration and Dashboard visualization
- Fraud management protocols
- Business Rules and Fraud detection mechanisms
- Threat detection and profiling methods
- Cost-benefit analysis for Big Data implementation for government
Introduction to Big Data
- Core characteristics of Big Data: Volume, Variety, Velocity, and Veracity
- MPP (Massively Parallel Processing) architecture
- Data Warehouses: static schema and slowly evolving datasets
- MPP Databases: Greenplum, Exadata, Teradata, Netezza, Vertica, etc.
- Hadoop-Based Solutions: flexible structure for datasets
- Typical processing pattern: HDFS, MapReduce (processing), retrieval from HDFS
- Apache Spark for stream processing
- Batch processing suitability for analytical/non-interactive tasks
- Volume management via CEP streaming data
- Typical selections: CEP products (e.g., Infostreams, Apama, MarkLogic, etc.)
- Less production-ready options: Storm/S4
- NoSQL Databases (columnar and key-value): optimal as analytical adjuncts to data warehouses/databases
NoSQL solutions
- KV Store - Keyspace, Flare, SchemaFree, RAMCloud, Oracle NoSQL Database (OnDB)
- KV Store - Dynamo, Voldemort, Dynomite, SubRecord, Mo8onDb, DovetailDB
- KV Store (Hierarchical) - GT.m, Cache
- KV Store (Ordered) - TokyoTyrant, Lightcloud, NMDB, Luxio, MemcacheDB, Actord
- KV Cache - Memcached, Repcached, Coherence, Infinispan, EXtremeScale, JBossCache, Velocity, Terracoqua
- Tuple Store - Gigaspaces, Coord, Apache River
- Object Database - ZopeDB, DB40, Shoal
- Document Store - CouchDB, Cloudant, Couchbase, MongoDB, Jackrabbit, XML-Databases, ThruDB, CloudKit, Prsevere, Riak-Basho, Scalaris
- Wide Columnar Store - BigTable, HBase, Apache Cassandra, Hypertable, KAI, OpenNeptune, Qbase, KDI
Varieties of Data: Introduction to Data Cleaning issues in Big Data
- RDBMS: static structure/schema, limiting agile, exploratory environments
- NoSQL: semi-structured, sufficient structure for data storage without pre-defined exact schema
- Data cleaning issues and considerations
Hadoop
- Criteria for selecting Hadoop
- STRUCTURED data: Enterprise data warehouses/databases can store massive data (at a cost) but impose structure (limiting active exploration)
- SEMI-STRUCTURED data: challenging to manage using traditional solutions (DW/DB)
- Data warehousing: significant effort and static nature even post-implementation
- For variety and volume of data, processed on commodity hardware: HADOOP
- Commodity Hardware required to create a Hadoop Cluster
Introduction to Map Reduce /HDFS
- MapReduce: distributing computing tasks over multiple servers
- HDFS: making data available locally for the computing process (with redundancy)
- Data: can be unstructured/schema-less (unlike RDBMS)
- Developer responsibility for interpreting data meaning
- Programming MapReduce: working with Java (pros/cons), manual loading of data into HDFS
Day 02
Big Data Ecosystem: Building Big Data ETL (Extract, Transform, Load) - Selecting appropriate Big Data Tools and timing
- Hadoop vs. Other NoSQL solutions
- For interactive, random access to data
- Hbase (column-oriented database) integrated with Hadoop
- Random access to data with specific restrictions (max 1 PB)
- Limitations for ad-hoc analytics; suitable for logging, counting, time-series
- Sqoop: Import from databases to Hive or HDFS (JDBC/ODBC access)
- Flume: Streaming data (e.g., log data) into HDFS
Big Data Management System
- Component mobility, compute node start/failure: ZooKeeper for configuration/coordination/naming services
- Complex pipeline/workflow management: Oozie – managing workflows, dependencies, daisy chains
- Deployment, configuration, cluster management, upgrades (sys admin): Ambari
- In Cloud environments: Whirr
Predictive Analytics: Fundamental Techniques and Machine Learning based Business Intelligence
- Introduction to Machine Learning
- Learning classification techniques
- Bayesian Prediction: preparing a training file
- Support Vector Machine
- KNN p-Tree Algebra and vertical mining
- Neural Networks
- Big Data large variable problem: Random forest (RF)
- Big Data Automation problem: Multi-model ensemble RF
- Automation through Soft10-M
- Text analytic tool: Treeminer
- Agile learning methods
- Agent-based learning
- Distributed learning
- Introduction to Open source Tools for predictive analytics: R, Python, Rapidminer, Mahut
Predictive Analytics Ecosystem and its application in Criminal Intelligence Analysis
- Technology and the investigative process
- Insight analytic
- Visualization analytics
- Structured predictive analytics
- Unstructured predictive analytics
- Threat/fraud/star/vendor profiling
- Recommendation Engine
- Pattern detection
- Rule/Scenario discovery: failure, fraud, optimization
- Root cause discovery
- Sentiment analysis
- CRM analytics
- Network analytics
- Text analytics for deriving insights from transcripts, witness statements, internet chatter, etc.
- Technology assisted review
- Fraud analytics
- Real Time Analytic
Day 03
Real Time and Scalable Analytics Over Hadoop
- Why common analytic algorithms fail in Hadoop/HDFS
- Apache Hama: for Bulk Synchronous distributed computing
- Apache SPARK: for cluster computing and real time analytic
- CMU Graphics Lab2: Graph-based asynchronous approach to distributed computing
- KNN p: Algebra-based approach from Treeminer for reduced hardware cost of operation
Tools for eDiscovery and Forensics
- eDiscovery over Big Data vs. Legacy data: a comparison of cost and performance
- Predictive coding and Technology Assisted Review (TAR)
- Live demonstration of vMiner for understanding how TAR enables faster discovery
- Faster indexing through HDFS: Velocity of data
- NLP (Natural Language processing): open source products and techniques
- eDiscovery in foreign languages: technology for foreign language processing
Big Data BI for Cyber Security: Getting a 360-degree view, speedy data collection and threat identification
- Understanding the basics of security analytics: attack surface, security misconfiguration, host defenses
- Network infrastructure / Large datapipe / Response ETL for real time analytic
- Prescriptive vs predictive: Fixed rule-based vs auto-discovery of threat rules from Meta data
Gathering disparate data for Criminal Intelligence Analysis
- Using IoT (Internet of Things) as sensors for capturing data
- Using Satellite Imagery for Domestic Surveillance
- Using surveillance and image data for criminal identification
- Other data gathering technologies: drones, body cameras, GPS tagging systems, and thermal imaging technology
- Combining automated data retrieval with data obtained from informants, interrogation, and research
- Forecasting criminal activity
Day 04
Fraud prevention BI from Big Data in Fraud Analytics
- Basic classification of Fraud Analytics: rules-based vs predictive analytics
- Supervised vs unsupervised Machine learning for Fraud pattern detection
- Business-to-business fraud, medical claims fraud, insurance fraud, tax evasion, and money laundering
Social Media Analytics: Intelligence gathering and analysis
- How Social Media is used by criminals to organize, recruit, and plan
- Big Data ETL API for extracting social media data
- Text, image, meta data, and video
- Sentiment analysis from social media feed
- Contextual and non-contextual filtering of social media feed
- Social Media Dashboard to integrate diverse social media sources
- Automated profiling of social media profile
- Live demonstration of each analytic capability via Treeminer Tool
Big Data Analytics in image processing and video feeds
- Image Storage techniques in Big Data: Storage solution for data exceeding petabytes
- LTFS (Linear Tape File System) and LTO (Linear Tape Open)
- GPFS-LTFS (General Parallel File System - Linear Tape File System): layered storage solution for Big image data
- Fundamentals of image analytics
- Object recognition
- Image segmentation
- Motion tracking
- 3-D image reconstruction
Biometrics, DNA, and Next Generation Identification Programs
- Beyond fingerprinting and facial recognition
- Speech recognition, keystroke analysis (analyzing a user's typing pattern), and CODIS (combined DNA Index System)
- Beyond DNA matching: using forensic DNA phenotyping to construct a face from DNA samples
Big Data Dashboard for quick accessibility of diverse data and display:
- Integration of existing application platform with Big Data Dashboard
- Big Data management
- Case Study of Big Data Dashboard: Tableau and Pentaho
- Use Big Data app to push location-based services in Govt.
- Tracking system and management
Day 05
How to justify Big Data BI implementation within an organization:
- Defining the ROI (Return on Investment) for implementing Big Data
- Case studies for saving Analyst Time in collection and preparation of Data: increasing productivity
- Revenue gain from lower database licensing cost
- Revenue gain from location-based services
- Cost savings from fraud prevention
- An integrated spreadsheet approach for calculating approximate expenses vs. Revenue gain/savings from Big Data implementation
Step-by-step procedure for replacing a legacy data system with a Big Data System
- Big Data Migration Roadmap
- Critical information needed before architecting a Big Data system
- Methods for calculating Volume, Velocity, Variety, and Veracity of data
- Estimating data growth
- Case studies
Review of Big Data Vendors and review of their products.
- Accenture
- APTEAN (Formerly CDC Software)
- Cisco Systems
- Cloudera
- Dell
- EMC
- GoodData Corporation
- Guavus
- Hitachi Data Systems
- Hortonworks
- HP
- IBM
- Informatica
- Intel
- Jaspersoft
- Microsoft
- MongoDB (Formerly 10Gen)
- MU Sigma
- Netapp
- Opera Solutions
- Oracle
- Pentaho
- Platfora
- Qliktech
- Quantum
- Rackspace
- Revolution Analytics
- Salesforce
- SAP
- SAS Institute
- Sisense
- Software AG/Terracotta
- Soft10 Automation
- Splunk
- Sqrrl
- Supermicro
- Tableau Software
- Teradata
- Think Big Analytics
- Tidemark Systems
- Treeminer
- VMware (Part of EMC)
Q/A session
Requirements
- Knowledge of law enforcement processes and data systems
- Basic understanding of SQL/Oracle or relational database
- Basic understanding of statistics (at Spreadsheet level)
Audience
- Law Enforcement specialists with a technical background
Testimonials (3)
basics and loved the prepared documents and exercises
Rekha Nallam - GE Medical Systems Polska Sp. z o.o.
Course - Introduction to Predictive AI
Deepthi was super attuned to my needs, she could tell when to add layers of complexity and when to hold back and take a more structured approach. Deepthi truly worked at my pace and ensured I was able to use the new functions /tools myself by first showing then letting me recreate the items myself which really helped embed the training. I could not be happier with the results of this training and with the level of expertise of Deepthi!
Deepthi - Invest Northern Ireland
Course - IBM Cognos Analytics
he was well prepared - and he is very sympathetic
