yomo-zipper
What is yomo-zipper?
yomo-zipper
is the workflow
in YoMo Ecosystem, you can use yomo-zipper
to manage the sequence of yomo-flow, and manage the yomo-sink which the real-time streaming data will output to.
What can yomo-zipper do?
yomo-zipper
receives the data from yomo-source, and transfers it to yomo-flow to run the real-time streaming computing. After the computing of each yomo-flow in pipeline, the data will be output to each yomo-sink in parallel.
yomo-zipper
connects the yomo-source
, yomo-flow
and yomo-sink
in YoMo Ecosystem, you can configure and run yomo-zipper
to experience the real-time streaming computing in YoMo. The data transmission is based on QUIC
and the high performance encoding and decoding by Y3 Codec.
How to config and run yomo-zipper?
Config yomo-zipper
The configuration of yomo-zipper
is a yaml
file, you can config the name
, host
and port
of yomo-zipper
, and add multiple yomo-flow
and yomo-sink
.
Example
name: Service
host: localhost
port: 9999
flows:
- name: Noise Serverless
host: localhost
port: 4242
sinks:
- name: Mock DB
host: localhost
port: 4141
Run yomo-zipper
1. Install CLI
# Make sure to use $GOPATH since golang requires the plugin and the main
# application to be highly coupled
$ echo $GOPATH
If $GOPATH
is not set, check here first: Setting $GOPATH and $GOBIN.
$ GO111MODULE=off go get -u github.com/yomorun/yomo
$ cd $GOPATH/src/github.com/yomorun/yomo
$ make install
2. Run yomo-zipper
Use the example in YoMo project:
$ yomo wf run example/workflow.yaml
2021/01/15 16:04:52 Running YoMo workflow...
2021/01/15 16:04:52 ✅ Listening on 0.0.0.0:9999
After running the above command, YoMo will start yomo-zipper
and listen on the specified port 9999
.
BTW: You can also run
yomo wf dev
to start yomo-zipper, it will automatically use the noise decibel value (from CELLA office) asyomo-source
, for example:yomo wf dev example/workflow.yaml
. In this case, you can ignore the step 4.
3. Run yomo-flow and yomo-sink
The following are using the examples in YoMo project, you can change them to the actual yomo-flow
and yomo-sink
which you want to run.
Run yomo-flow
:
$ yomo run example/flow/app.go
2021/01/15 16:10:13 Building the Serverless Function File...
2021/01/15 16:10:21 ✅ Listening on 0.0.0.0:4242
Run yomo-sink
:
$ yomo run example/sink/app.go -p 4141
2021/01/15 16:13:29 Building the Serverless Function File...
2021/01/15 16:13:31 ✅ Listening on 0.0.0.0:4141
YoMo also provides the following examples for yomo-sink
, you can choose one to experience a more realistic scene.
- yomo-sink-faunadb-example provides an example of storing the data in
FaunaDB
. - yomo-sink-socketio-server-example provides an example of receiving the real-time noise decibel value and use
socket.io server
to broadcast it toWeb
page. - yomo-sink-socket-io-example provides an example of receiving the noise decibel value from
socket.io server
and show it on aReact
page.
4. Run yomo-source
Use yomo-source-example as an example to send the mocking noise decibel value. You can change it to the actual yomo-source
which you want to run.
Download yomo-source-example to local and run the following command:
$ go run main.go
2021/01/15 16:18:10 ✅ Connected to yomo-zipper localhost:9999
2021/01/15 16:18:10 ✅ Emit 119.512955 to yomo-zipper
2021/01/15 16:18:11 ✅ Emit 82.569893 to yomo-zipper
2021/01/15 16:18:11 ✅ Emit 160.101456 to yomo-zipper
2021/01/15 16:18:11 ✅ Emit 170.802765 to yomo-zipper
2021/01/15 16:18:11 ✅ Emit 86.156288 to yomo-zipper
2021/01/15 16:18:11 ✅ Emit 114.443230 to yomo-zipper
2021/01/15 16:18:11 ✅ Emit 17.846315 to yomo-zipper
2021/01/15 16:18:11 ✅ Emit 166.903183 to yomo-zipper
YoMo also provides the following example:
yomo-source-mqtt-broker-starter provides an example of receive MQTT
messages and connect to YoMo over QUIC
. If you are using IoT devices based on the MQTT
protocol, you can refer to this example to connect the MQTT messages as a yomo-source
.
5. yomo-zipper receiving data and performing real-time streaming computing
2021/01/15 16:04:52 ✅ Listening on 0.0.0.0:9999
2021/01/15 16:18:10 ✅ Connect to Noise Serverless (localhost:4242) successfully.
2021/01/15 16:18:11 ✅ Connect to Mock DB (localhost:4141) successfully.
Optional: Setting $GOPATH and $GOBIN
For the current session only:
export GOPATH=~/.go
export PATH=$GOPATH/bin:$PATH
To permanently set these variables, you need to edit .zshrc
or .bashrc
:
For zsh
users:
echo "export GOPATH=~/.go" >> .zshrc
echo "path+=$GOPATH/bin" >> .zshrc
For bash
users:
echo 'export GOPATH=~/.go' >> .bashrc
echo 'export PATH="$GOPATH/bin:$PATH"' >> ~/.bashrc