Jai’s Weblog – Tech, Security & Fun…

Tech, Security & Fun…

  • Jaibeer Malik

    Jaibeer Malik
  • View Jaibeer Malik's profile on LinkedIn
  • Subscribe

  • Feedburner

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 29 other followers

  • Archives

  • Categories

  • Stats

    • 382,329
  • Live Traffic

IOT Security: Attack surface areas, Vulnerabilities & Considerations

Posted by Jai on January 5, 2017

In this post we will discuss the security aspects around connected devices solution. As typically goes in M2M information flow, from connected devices hardware till the information is available to end-user on dashboard or mobile gadget.

There are varieties of connected devices getting in market day by day. The size and processing power of each of such devices put extra emphasis on the security aspects around these devices. Here we will discuss the information flow in case of connected devices, how data flows from devices to end users of those devices. On top of that what are different attack surface areas, what common vulnerabilities those expose and what are the common ways to eliminate those threats for connected devices.

In the previous posts, we discussed about availability of different type of IOT platforms in market, typical characteristics of such a platform and the technical challenges in building your own IOT platform.

IOT Platform: Typical Characteristics
Building IOT platform, Technical challenges

Below diagram explains the typical IOT data flow, possible vulnerabilities is such system and the considerations to build strong and secure security architecture model for such solution.


Information Flow Networks

In a typical IOT scenario, the device captures the data, either using local gateway or telecom the data is transmitted to cloud server for further processing. The processed data is further made available to end consumer or respective events are triggered or information shared further with ecosystem parties.

Commonly below different data flow networks are used in the system,

  1. Device Network
  2. Device Communication Network
  3. Telecom Network
  4. Internet
  5. Hosting/Cloud environment Network
  6. IOT Platform
  7. Mobile applications/Gadgets network
  8. Data Privacy

Read the rest of this entry »

Posted in IOT, Security | Tagged: , , , | Leave a Comment »

Building IOT platform, Technical challenges

Posted by Jai on January 2, 2017

As discussed in the last post regarding the typical characteristics of IOT platforms, there are different categories of available IOT platform these days and typical feature and functional support provided by each. Device connectivity, device agnostic, domain agnostic and extensibility etc. drive to come up with few required technical aspects of such a platform. In this post, we will discuss different technical aspects to be taken into account to build such IOT platform.

Typical characteristics of any IOT platform are as covered in last post.

In general, we can add below typical technical aspects in the architecture of such IOT platform


  1. Device Connectivity
  2. Device repository and management
  3. Data Processing of Events/Rules Management
  4. Alerting, Monitoring & Notification support
  5. Complex Event Processing support
  6. Scalable & Extensible Data storage layer
  7. Real time Data access support
  8. Data Analytics
  9. Data Visualisation support
  10. External Ecosystem system Integration support
  11. IOT Security, Across all layers
  12. Development/Testing Support
  13. Testing/POC/Time to Market support
  14. Device Bi directional support
  15. Device Firmware management support
  16. Data reporting/export/publish/share support

Read the rest of this entry »

Posted in IOT | Tagged: , , , | 1 Comment »

IOT Platform: Typical Characteristics

Posted by Jai on June 29, 2016

There are common characteristics for iot platforms but it is indeed very difficult to benchmark iot platform solution as many of these try to solve fundamentally different problems. In this post we will cover some of the typical characteristics which such a solution should have.

IOT Platform Categorisation

IOT Platforms

IOT Platforms

There are lot of IOT platform providers now a days and numbers are growing every year. Each having some very common features and some having quite different features. On high level, we can categories these IOT platforms in different categories,

Connectivity / M2M platforms

To provide connectivity to connected devices with cloud.

IaaS backends

Infrastructure-as-a-service backends provide hosting space and processing power for applications and services.

Hardware-specific software platforms

Domain specific platform supporting only those connected devices.

Consumer/Enterprise software extensions

Existing enterprise software packages and operating systems allowing the integration of IoT devices.
Read the rest of this entry »

Posted in IOT | Tagged: , , , , , | 2 Comments »

HBase: Generating search click events statistics for customer behavior

Posted by Jai on July 9, 2014

In this post we will explore HBase to store customer search click events data and utilizing same to derive customer behavior information based on search query string and facet filter clicks. We will cover to use MiniHBaseCluster, HBase Schema design, integration with Flume using HBaseSink to store JSON data.

In continuation to the previous posts on,

We have explored to store search click events data in Hadoop and to query same using different technologies. Here we will use HBase to achieve same,

  •  HBase mini cluster setup
  •  HBase template using Spring Data
  •  HBase Schema Design
  •  Flume Integration using HBaseSink
  •  HBaseJsonSerializer to serialize json data
  •  Query Top 10 search query string in last an hour
  •  Query Top 10 search facet filter in last an hour
  •  Get recent search query string for a customer in last 30 days


Read the rest of this entry »

Posted in Architecture, Flume, Hadoop, HBase, Java, Spring Data | Tagged: , , | 1 Comment »

Spark: Real time analytics for big data for top search queries and top product views

Posted by Jai on June 4, 2014

Hadoop being the batch processing framework makes it a little hard to get the real time analytics for big data. Apache Spark overcomes this batch nature and provides distributed computation capabilities and events processed in streaming fashion. In this post, we will cover to explore Spark streaming capability to process Flume Events data to generate Top search query strings generated in last an hour or top product views in the last one hour.

In continuation to the previous posts on

We have so far utilized the Hadoop system batching capabilities to process huge amount of data. But the overall batching operation makes it a bit of latency issue depending on your data. This is where Spark comes into picture. We will explore Spark streaming capability here to get some real time analytics and those can be used on the website for display purpose or for monitoring purpose.


Apache spark  “is a fast and general engine for large-scale data processing.”


As shared in other above exmaples, we have the customer search clicks data available to us. We have Flume system in place to process the data and store in Hadoop for later processing perspective. Take a scenario, you want to display real time customer behavior on the website, how other customers are doing

  • What other customers searching?
  • Other customers also searching for…
  • Top search query string on the website in last an hour
  • What other customers viewing?
  • Other customers also viewing products…
  • Top product views in the last an hour

Read the rest of this entry »

Posted in Flume, Hadoop, Java, Spark | Tagged: , , , , , | 2 Comments »

%d bloggers like this: