Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼

Tooling Up for the Marriage of the Internet of Things, Big Data, and Cloud Computing

The inexorable march of technology has the computing and software community focusing on a large-scale integration challenge by exploiting cloud computing, machine-to-machine (M2M) applications, and Big Data solutions. The convergence of M2M, Big Data, and the cloud will provide key capabilities for building next-generation systems and expanding the Internet of Things. This Internet of Things (IoT) will sustain communication and data sharing for billions of connected devices and systems. The number of connections and endpoints is so large it requires IPv6 addresses because the IPv4 addressing scheme cannot accommodate the volume of sensors, smart phones, smart factories, smart grids, smart vehicles, controllers, meters, and other devices that will be transmitting data over the Internet.

The sheer volume of data from human users and M2M applications will require advanced analytics capable of exploiting the Big Data and the computing power of the cloud. There are ambitious projects being done in the IoT/M2M realm, including a pipeline of R&D projects from Planetary Skin, a planetary-monitoring effort that can provide global data collection using space-based and earth-based sensor networks. It's no surprise the convergence of IoT, Big Data analytics, federated databases, and other technologies is now being called the Internet of Everything (IoE).

Today's Internet of Things phenomenon is a reminder of the XML and Web services explosion, with partnerships and consortia pushing to develop specifications and submit them to standards organizations. If you write software that follows the specification for a standard, such as a communication protocol, you increase the chances it will be interoperable, won't create data silos, and will be accepted by the marketplace. Among the consortia working on standards for IoT are AllSeen Alliance, HyperCat Consortium, and Industrial Internet Consortium. There are also initiatives such as the Eclipse M2M Industry Working Group and ITU-T Focus Group M2M initiative. And of course technology companies such as Apple, Cisco, Google, IBM, Intel, Microsoft, and Oracle are competing in the IoT game. The hope is that all of these efforts will deliver something like a universal plug-and-play solution, with a common service capability layer, that will simplify programming and enable devices to "learn" how to seamlessly connect to other devices and machines. Everyone would also like to see trusted mobility computing with managed, secure application programming interfaces (APIs), and a standard framework for mobile device security.

The Integration Challenge

Building the IoT will be an exercise in integrating disparate devices and carrier networks, multiple communication protocols, and a wide variety of applications. It will often require integrating legacy networks and applications. The new M2M applications are and will continue to be complex, using geographically dispersed devices and services, a mix of connectivity, and logic in the data center and edge devices.

How to handle that diversity is one of today's major challenges. Current practices are similar to the early years of software development when everything was done as an ad hoc application before the value of data sharing was widely accepted. Large-scale IoT projects, such as building a smart city, will not be successful using the fragmented, domain-specific approach of M2M solutions designed for specific M2M applications. Successful large-scale IoT/Big Data/Cloud implementations require an integration focus and a data pipe that can move massive amounts of data from heterogeneous sources. Ingredients the architect should use to bake an IoT cake include dynamic discovery of devices and objects, on-the-fly reconfiguration capabilities, policy-driven provisioning and operation, and software-defined networking. The icing on the cake will be APIs for data analytics and leveraging investment in existing infrastructure.

A typical M2M architecture includes an application domain, a network domain, an M2M device domain and one or more direct connections or gateways from the M2M area network to the network domain. Because M2M device area networks can use a variety of communication technologies (RFID, ZigBee, M-BUS, IEEE 802.15, 6LoWPAN), a gateway layer becomes important. The solutions for communication between the gateway and M2M applications include LTE, WiMAX, xDSL, and WLAN. In the application domain, clients will often include dashboards for data virtualization, status monitoring, reconfiguration and other functions.

With this M2M architecture, roles span layers. There are service providers and application providers that run services that a device owner consumes. The transport network owner's customer is the service provider, and there are also device or gateway operators. Role and identity are critical for provisioning, access to resources, and rights to data.

M2M Security

Because security is not a major achievement of today's Internet, scaling up to 50- to 200-billion connected devices requires innovative solutions. System architects must focus on security at endpoints and when data is in transit: device security, cloud security, and network security. Virtualization must be done with secure virtual machines (VM), secure single-tenant and multi-tenant servers, and secure software defined networks (SDN).

The challenges include the security of smart devices, M2M devices in area networks, and interconnecting networks. The hacking of traffic control systems, baby monitors, smart TVs, and industrial control systems (ICS) has shown us that the threats to connected devices are pervasive. One requirement is authentication, such as in unlocking the doors of a vehicle. In other instances, the requirement is protecting confidential data, such as financial credentials, moving across the network pipe.

We've moved past an era when a perimeter defense against external threats is adequate. Recently, for example, a smart phone manufactured in China was found to have spyware embedded in its firmware. The architecture of the IoT must embrace multi-level security with end-to-end protection. It must support secure communication with data encryption, using standard protocols such as Transport Layer Security (TLS). It should include the use of security processors or encryption chips, and even use lightweight encryption for low-bandwidth devices. We should also exploit smart cards for storing keys and encryption algorithms and SMS authentication by operators. Some SIM makers, such as Gemalto, are developing MIMs that provide authentication and encryption to ensure the identity of machines.

SCADA and Industrial Control Systems

SCADA (supervisory control and data acquisition) systems arrived several decades ago, before the emergence of the Internet, so their architectures did not focus on secure connectivity to a global network. Over time, Internet connections became a fact of life for SCADA and industrial control systems (ICS), but they were largely insecure connections. Legacy SCADA systems were vulnerable to insider threats, but the Internet added threats such as denial-of-service, malicious firmware upgrades, identity spoofing, and execution of risky commands.

Stuxnet raised our awareness of cyber threats to industrial control systems. We've recently learned of hacker crews targeting more than 1000 energy companies in 84 countries, including ICS penetrations. And we now know that ICS/SCADA systems in Europe were attacked by botnets that distributed the Havex Remote Access Trojan (RAT).

Establishing machine and object identity can mitigate many risks. For example, Verizon offers an authentication solution: a Managed Certificate Services (MCS) platform for verifying machine identities and securing data transmissions. Establishing identity provides a means for managing data access rights and establishing white and black lists on servers.

M2M and IoT integrity is diminished when security and privacy is an afterthought, grafted on instead of built in. The firmware in connected devices should have an expiration date by which it must be updated. M2M applications will often involve supporting a multi-vendor ecosystem, with connections over Bluetooth, Ethernet, and WiFi. End-to-end security is integral to the design of the Internet of Things and will be a key factor in successful implementations.

M2M/IoT Tools

Development of M2M applications has typically been performed with a variety of protocols using tools from disparate vendors. But now there is growing emphasis on standard protocols, interoperability, tools, and M2M/IoT integration platforms.

Integration platforms will become much more important as we experience the convergence of IoT, cloud computing, analytics, and Big Data. Besides supporting links among cloud applications, they will increasingly become a solution for tying together the distributed devices at one end of a network pipe with enterprise applications and analytics at the other end. Integration platforms shorten the development cycle for connecting devices to the cloud or enterprise systems.

The tools in this space were traditionally built to simplify development with specific, targeted devices and protocols, but now there's interest in tools with broader applicability. The Eclipse Foundation is pushing development of IoT and M2M applications using open-source software. Its efforts include a set of tools and libraries (Koneki), a Linux-based embedded application framework (Mihini), and a collection of communication protocols (Paho). Multiple companies offer toolkits for developers building wireless M2M applications. These include the AT&T M2M Application Platform, the Deutsche Telekom M2M Toolkit, and the Sprint M2M Toolkit. Another alternative is to use an M2M bundle, such as those from Vodafone and other providers. The Vodafone Integrated M2M Terminals, for example, bundle pre-configured connectivity services and hardware, such as smart meters.

There are gateways and integration solutions in existence for bringing together the disparate technologies to be found in a complex M2M project. They act as an intermediary between distributed devices and the applications and services that operate on the data, such as analytics.

Building on a Platform

A project that requires widespread connectivity, security, and the successful marriage of M2M applications and cloud computing is an ambitious undertaking. For many of us, it would be the most complex challenge we've faced. For that reason, we'll naturally favor the use of a platform that handles many of the integration problems of a geographically dispersed, multi-vendor, cloud environment.

For example, one such platform from M2Mi provides a solution for secure software-defined networking (SDN) and rapid provisioning despite the challenges of heterogeneous, multi-vendor interoperability. An effective SDN solution requires device abstraction, dynamic discovery of network topology, and intelligent orchestration. M2Mi Software Defined Networking can provide a secure virtual network for building a dynamic, cloud-based, services-delivery platform, managing orchestration among disparate network and security providers. It uses MQTT as a bridging protocol, complies with established security standards and guidelines (such as the NIST Cybersecurity Framework), and can support combinations of security policies and firewalls.

M2Mi SDN can manage both single-tenant and multi-tenant clouds, and it provides libraries that support new and legacy networking. It performs dynamic auto-discovery of network topology, as shown in Figure 1. M2Mi SDN can discover the topology for a set of servers, identify outages, device configurations and correct port settings, and assign provisioning roles.

Figure 1.

The process of creating a secure software-defined network configuration includes creating user accounts and rights, specifying security zones with their rules, and using defining templates for provisioning. Creating an application template is similar to creating a VM template.

In this context, orchestration is the sequencing of actions that enable devices and applications to undertake complex tasks. Orchestration provides abstraction of API and service invocations between devices and applications. After an application or VM is provisioned in a security zone, M2Mi SDN Orchestration configures each security object and policy required for the VM or application to operate appropriately within the zone. It also provides the ability to rapidly change the network and service topology. This is the type of platform we'll need to face the Internet of Everything with sanity.


The convergence of M2M/IOT, cloud computing, and Big Data technologies is both an opportunity and a challenge. These technologies give us the engine for a powerful new generation of services and applications. But there will be trials, such as overcoming complexity and security threats, which are best addressed by adhering to standards and using powerful tools for integration and interoperability.

Ken North is a well-known expert in database technologies and long-time contributor to Dr. Dobb's.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.