Zhiqing's Blog

How to use AMQP protocol in Browser (JavaScript)

michaeljqzq — Tue, 28 Mar 2017 03:32:32 GMT

In my previous 2 posts, Connect to Azure Event Hub in browser ( using AMQP over WebSockets ) and Connect to Azure IoT Hub in browser ( using AMQP over WebSockets ), I introduced the parameters needed to connect to IoT Hub and Event Hub. Since we chose to connect in front-end, so we have to find a JavaScript library for AMQP. However, it isn't quite easy to find one. rhea is a reactive messaging library based on the AMQP protocol which meet our need.It is a NodeJS project, but it provides a browserify command in script. So all we have to do is to make a little changes. 1.In the type wrapper (types.js), it is designed for Node so it uses Buffer class in code. [javascript] if (o instanceof Buffer) { return types.wrap_binary(o); } [/javascript] The corresponding data structure in front-end JavaScript is ArrayBuffer. So here we make this code compatible with browser. [javascript] if (o instanceof Buffer || o instanceof ArrayBuffer) { return types.wrap_binary(o); } [/javascript] 2.The library doesn't implement disconnect method. So let's add one in container.js [javascript] Container.prototype.disconnect = function () { this.connection.close(); }; [/javascript] Now let's try to browerify the project! First we need to set dependency.In the terminal, point to project root folder, set the environment variable NODE_PATH to the parent folder of your rhea project. For example, my project folder is C:\Users\v-zhq\IoT\rhea then you will set NODE_PATH to C:\Users\v-zhq\IoT. Then execute in project folder C:\Users\v-zhq\IoT\rhea:

npm run browserify

rhea.js get! Then you can use example to test! Point to examples/websockets and start an HTTP server. Then visit https://localhost:8000, input websocket url. client.html ( in line #31 I made a little change or client object can't be initialized ) [html] AMQP websockets example [/html]

使用MQTT协议往Android手机推送消息

michaeljqzq — Sun, 19 Mar 2017 06:39:07 GMT

最近做了一些IoT相关的工作，对于MQTT协议用得比较多。MQTT诞生较早（1999年）但是一直不温不火，直到最近由于物联网的兴起，才由IBM在2013年将其提交到了OASIS规范中，从而在物联网的协议中占得一席之地。这个协议有着如下特点：

轻量化，消息头部采用二进制格式

订阅/发布模式，这样非常适合于服务器推送

可以对不同的话题采用不同的QoS（服务质量），可以在可靠性和复杂度之间取得平衡

介于之前一直想做一个往自己Android手机上推送消息的小应用。但是因为某些网络环境的原因~~你懂的~~，Google为Android设计的推送机制在国内并不能很好的使用。如果使用类似于Rest API轮询的方式，对于移动设备来说过于耗费电量以及流量，并不是很好的方法。了解到有了MQTT这种轻量级的协议以后，发现完美契合这个推送消息的需求。于是赶紧试一试！在整个流程中，需要3个点，分别是MQTT Broker,推送消息的client，以及接收消息的Android client。 1. MQTT Broker 就是一个接受订阅，接受并转发消息的服务器。我选择基于Node的项目mosca，由于我是独立使用，所以特别简单。

npm install mosca pino -g
mosca -v | pino

简直不要太方便！查看一下网络端口

tcp6       0      0 :::1883                 :::*                    LISTEN

好，已经默认监听在1883端口了。 2. 推送消息的client 就是要推送消息的消息源。这个还是用Node的MQTT项目（Node的库真是好用哭）。同样是两种用法，第一种独立使用：

npm install mqtt -g
mqtt sub -t 'topic' -h 'broker_url' -v
mqtt pub -t 'topic' -h 'broker_url' -m 'message'

第二种嵌入code： [javascript] var mqtt = require('mqtt') var client = mqtt.connect('mqtt://test.mosquitto.org') client.on('connect', function () { client.subscribe('presence') client.publish('presence', 'Hello mqtt') }) client.on('message', function (topic, message) { // message is Buffer console.log(message.toString()) client.end() }) [/javascript] 3. 接收消息的Android client 这个比前两个找起来难度大点。在Play市场上搜索MQTT，试了一些排名靠前的应用，发现对于MQTT的支持都没有什么问题，收发消息也都正常。但是都无法支持后台保持连接以及消息提醒机制。最后找到了这么一个应用，可以实现MQTT消息的后台以及息屏提醒，叫做Linear MQTT Dashboard 打开应用先进行一下简单的设置返回到主界面，可以看到有两个状态指示灯第二个就表示MQTT Broker的连接状态，如果是绿色的就说明已经成功连接上了。然后增加一个widget来显示消息，先按一下上图中的暂停按钮，然后按加号添加一个widget，其中主要填好Sub.topic就可以了我们试着从另一个地方往test话题推送一条消息

mqtt pub -t 'test' -h '[MQTT_BROKER_URL]' -m 'hello world'

接收成功，那么看看如果app切到后台能不能收到提醒呢可以看到，成功在通知栏提醒了消息，并且有震动。在息屏时也会有震动提醒，完美！那么到此为止，就利用MQTT协议成功的进行了从任意client往Android手机推送消息的机制，而且非常省电和省流量。其实在这个方案中，Android手机不仅可以作为消息的接收端，也可以作为消息的发送端，那么如果把接收端放在一些嵌入式设备或者小型开发板上（由于MQTT是物联网的标准协议，所以支持较为广泛），那么就可以通过手机来控制这些设备，进而实现智能家居等等任何你能想象的功能！

Connect to Azure Event Hub in browser ( using AMQP over WebSockets )

michaeljqzq — Thu, 16 Mar 2017 12:52:45 GMT

In my last post, I introduced the way to connect to Azure IoT Hub using AMQP over WebSockets. At the end of post, I said through this way we can only send C2D message to devices. However, open receiver to topic:/messages/serviceBound/feedback won't get any message when device send D2C message. So here's the problem, how to receive D2C message? The answer is: Event Hub. You may ask, why can we get IoT Hub message through Event Hub? If you are familiar to Azure IoT Hub, you may find there's a "compatible" Event Hub in IoT Hub portal: You may guess that IoT Hub is based on Event Hub? I had the same inference, but after contacting Azure suppport, they told me IoT Hub doesn't based on Event Hub.However, since IoT Hub are similar to Event Hub in many ways(both have message queue and handler), Azure provides a compatible way to visit IoT Hub as an Event Hub.So IoT Hub is not based on Event Hub, but compatible with Event Hub. So now we can focus on how to fetch message from Event Hub.Let's go! First, we must establish a websocket channel.

wss://[EventHubEndpoint]:443/$servicebus/websocket
Sec-WebSocket-Protocol: AMQPWSB10

Then you will have to establish AMQP connection. Use the following parameters to connect, you need to use SASL.

Key	Value
hostname	[EventHubEndpoint]
container_id	"conn" +
max_frame_size	4294967295
channel_max	65535
idle_timeout	120000
outgoing_locales	en-US
incoming_locales	en-US
offered_capabilities	null
desired_capabilities	null
properties	{}
connection_details
reconnect	false
username	[accessPolicyName]
password	[sharedAccessKey]

After connection opened, we will open receiver on all partitions in Event Hub. How do we get all partition ids in Event Hub? We create a sender and a receiver on topic:

$management

And then we will send a message on topic $management:

body:'[]',
                application_properties: { 
                    operation: 'READ',
                    name: [EventHubPath],
                    type: 'com.microsoft:eventhub' 
                }

Then we will receive a meesage with partition info:

partition_count:2,
partition_ids:[0,1]

Then we will open several receivers to fetch message.( The number of receivers equals to partition count ）

/[eventHubPath]/ConsumerGroups/[consumerGroup]/Partitions/[partitionId]

That's all for the work! Sneezry and I just finished a Web Project including Device/Service/Registry(device management).If you feel a bit confusion about the complicated process above, you can try this tool and it will provide you a easy-access experience to Azure IoT Hub!

Connect to Azure IoT Hub in browser ( using AMQP over WebSockets )

michaeljqzq — Tue, 14 Mar 2017 05:10:53 GMT

Azure provides simple ways to connect to IoT Hub through azure-iot-sdk, but some developers would like to figure out what the sdk exactly did or integrate with their own services. In this situation, it is necessary to find out how to connect to Azure IoT Hub above protocol level. AMQP is the only protocol to connect to IoT Hub ( You can find detailed spec of this protocol here, but trust me, it isn't easy to go through this ). Since using browser to do this is quite convenient for developing and debugging, so we will choose AMQP over WebSockets to connect. First we have to create a websocket tunnel. Here is the url:

wss://.azure-devices.net:443/$iothub/websocket

Sec-WebSocket-Protocol: AMQPWSB10

Then you will have to establish AMQP connection. Use the following parameters to connect, you need to use SASL.

Key	Value
hostname	.azure-devices.net
container_id	"conn" +
max_frame_size	4294967295
channel_max	65535
idle_timeout	120000
outgoing_locales	en-US
incoming_locales	en-US
offered_capabilities	null
desired_capabilities	null
properties	{}
connection_details
reconnect	false
username	@sas.root.
password

There are two ways to generate SAS. a.Generate by Device Explorer. Device Explorer is a tool provided by Azure in Windows environment. You can use it to generate SAS token. First, get IoT Hub connection string from portal Paste connection string in Device Explorer. In the same tab, set TTL and click "Generate SAS", you will get SAS token. b.Calculate SAS To save your time and energy, DON'T use this way if you can generate one using above method. DON'T use this way if you don't want to investigate a bunch of things about Hash algorithm, signature,etc... DON'T use this way if you don't have enough reason to do this. OK. It seems you are still interested in SAS, let's do this!

SharedAccessSignature sig=&se=&skn=&sr=
se: expiration time of this token
skn: role in shared access policies
sr: your iot hub url
sig: the SHA-256 hash of the scope of the token with a CRLF appended and an expiry time.

sig = hmacHash(_key, sr + '\n' + se) // _key is a base64-encoded key value

After connection is established and opened, we need to open a receiver and a sender. We use sender to send cloud-to-device message and use receiver to receive feedback message.(Unfortunately, currently we didn't find a way to fetch device-to-cloud message, we guess we can only do this by connecting EventHub or ServiceBus)

Sender:/messages/devicebound
Receiver:/messages/serviceBound/feedback

OK, now we're finished! You can test with your device to see if it works. If you don't have a device in hand or feel troublesome to use device, you can try our Web-based Iot device simulator. It can simulator an Iot Device and communicate with IoT Hub. | Connect to Azure IoT Hub in browser ( using AMQP over WebSockets )

Use Fiddler to capture websocket packet within non-browser runtime (like Node.js)

michaeljqzq — Thu, 09 Mar 2017 03:29:42 GMT

If you would like to skip the zig-zag and find the answer quickly, click me Recently I'm involved in a project which needs to analyze packet of high-level protocol( MQTT , AMQP ) over Websocket. It is quite easy to capture Websocket packet from browser. Chrome has some support for WS packet capture. However, high-level protocol send Binary data frame, and we can't easily see the binary content. Fortunately, Fiddler can help us! Fiddler will show a tab "WebSocket" and we can easily see the full content of packets in Websocket like this: But how to capture packet which doesn't come from browser, but from other runtime (for example Node.js)? I tried npm config first,


npm config set proxy https://127.0.0.1:8888
npm config set https-proxy https://127.0.0.1:8888

It seems that this proxy setting only works for requests between client and npm server. And also tried ways in this answer on SO, but also no luck. Finally I found this fantastic tool,Proxifier. According to its description, it allows network applications that do not support working through proxy servers to operate through a SOCKS or HTTPS proxy and chains.

Here's the detailed steps:

1.Configure two settings in Proxifier a.Proxy server setting: Profile->Proxy server , set Fiddler proxy server here.

b.Rule setting: We have to configure a few rules:

set Fiddler.exe traffic not go through proxy, or the will be infinite loop
set your taget application go through proxy

2. If your application use websocket over TLS/SSL, the self-signed certificate provided by Fiddler will be rejected. Trust Fiddler root certificate in system CA list doesn't work.(It seems that Node maintains its own trusted list) So we can configure an environment variable to solve this. For Windows CMD: [bash] set NODE_TLS_REJECT_UNAUTHORIZED=0 [/bash] For Windows Powershell: [powershell] $env:NODE_TLS_REJECT_UNAUTHORIZED = 0 [/powershell] For unix: [bash] export NODE_TLS_REJECT_UNAUTHORIZED=0 [/bash] Run your application and enjoy!