Exploring Enterprise Search Solution Critical Capabilities

In this series of blog posts we will review the enterprise search solution capabilities, available software solutions, from basic to advanced search capabilities along with using AI/ML models to achieve both consumer and business value. We will also cover the search solution for healthcare domain common user interactions, domain dataset and ML alignments. In this post we will cover the critical enterprise search solution capabilities on high level.

History

A long has changed in the search solution capabilities since the initial blog posts shared here in this blog for ElasticSearch capabilities. Additionally using Elasticsearch for user behavior clickstream data with Hadoop big data capabilities to process those information. A quick glance at those here,

ElasticSearch

Getting used to Elastic search and having hands directly with different tricks,

Getting started with ElasticSearch

ElasticSearch: Faceted Search for Hierarchical data

ElasticSearch: Text analysis for content enrichment

ElasticSearch: Boosting score for content relevancy

ElasticSearch: Learn Java API usage with test cases

ElasticSearch: Indexing setup using Akka tutorial

Car Dash Cam: How secure is your data?

With car dash cam, all information inside your car is recording and shared on cloud across internet and accessed on apps. How secure is your data with any dash cam system is what we will discuss in this post covering different aspects of data for such a solution.

Few aspects of car dash cam which we already covered,

Why do you need a Dash Cam?

Choosing the best car dash cam for your needs?

Cam Security Breaches?

It is just matter of time, sooner or later it happens if not enough thought given behind it.

To secure consumer data should be first and foremost responsiblity of each company and sooner they realize sam ebetter it is for both.

Somebody’s Watching: Hackers Breach Ring Home Security Cameras

What all Dash Cam Data?

What all data is captured and stored by car dash cam solutions. Once mounted in your cam, your dash cam recording everything happening inside and outside your car. There is lot of personal information recorded and stored and transferred over the wire.

HBase: Generating search click events statistics for customer behavior

In this post we will explore HBase to store customer search click events data and utilizing same to derive customer behavior information based on search query string and facet filter clicks. We will cover to use MiniHBaseCluster, HBase Schema design, integration with Flume using HBaseSink to store JSON data.

In continuation to the previous posts on,

• Customer product search clicks analytics using big data,
• Flume: Gathering customer product search clicks data using Apache Flume,
• Hive: Query customer top search query and product views count using Apache Hive,
• ElasticSearch-Hadoop: Indexing product views count and customer top search query from Hadoop to ElasticSearch,
• Oozie: Scheduling Coordinator/Bundle jobs for Hive partitioning and ElasticSearch indexing,
• Spark: Real time analytics for big data for top search queries and top product views

We have explored to store search click events data in Hadoop and to query same using different technologies. Here we will use HBase to achieve same,

•  HBase mini cluster setup
•  HBase template using Spring Data
•  HBase Schema Design
•  Flume Integration using HBaseSink
•  HBaseJsonSerializer to serialize json data
•  Query Top 10 search query string in last an hour
•  Query Top 10 search facet filter in last an hour
•  Get recent search query string for a customer in last 30 days

Svn2Git: Migrating repository from Subversion to Git

Efficient enough Version Control System has always been a challenge based on your requirements of local, centralized or distributed version controlling. Git a distributed version controlling system helps us to achieve same painlessly and is already out for quite some time now with proven track record. In this post we will cover the steps to migrate from your existing version control system like SVN to Git.

Why Git

Some of the features of Git which makes it to standout,

• Fixed the pitfalls and learning from SVN
• Dramatic increase in operation speed (diff, merge, view history etc.)
• Easy, Cheap and efficient branch operations
• Full history tree available offline
• Distributed, peer-to-peer model
• Git’s repositories are much smaller than Subversions
• Git branches carry their entire history
• Git provides better auditing of branch and merge events
• Git’s repository file formats are simple, so repair is easy and corruption is rare.
• Backing up Subversion repositories centrally is potentially simpler – since you can choose to distributed folders within a repo in git
• Git repository clones act as full repository backups
• Walking through versions is simpler in Subversion because it uses sequential revision numbers (1,2,3,..); Git uses unpredictable SHA-1 hashes. Walking backwards in Git is easy using the “^” syntax, but there is no easy way to walk forward.

ElasticSearch: Faceted Search for Hierarchical data

Faceted Search is the navigational search allowing business to clearly define the properties or characteristics of the product catalog and navigate user to find relevant products with minimum efforts. Most of the available search solutions support the functionality now a days, in this post we will cover how to implemented faceted search using flattened data approach for hierarchical data using ElasticSearch for a typical eCommerce platform.

Search Scenarios/Business Example:

Earlier post, Data Modeling approach for search content and tagging  explains the different characteristics of a typical eCommerce platform serving hierarchical data in terms of categorization of data and sub categorization.

Take an example of such a typical eCommerce platform where one site you need to display the Navigration browsing of your hierarchical data based on some search solution. For example, you need to display products like Books/Clothes etc. Each product has its own specific characteristics and can be categoriezed in different categories and sub categories.

Hierarchical Data:

The hierarchical data in business form represents the taxonomy for your data. The way you can characterize your data in the form of different category type, categories and sub categories for the product catalog.