For Developers, By Developers

Explore Streamdata.io and learn about the ins and outs of our realtime streaming proxy.

cell phone with data displayed on interface

Developers

Get to Know Streamdata.io

Without robust client-side integration tools, Streamdata.io provides a proxy-as-a-service that instantly turns any request/response API into a realtime event-driven data feed. And because it can enhance data provider operations, user retention through stunning interfaces, and data acquisition for machine learning models, we see Streamdata.io as a solution for businesses across all industries.

More Ways To Learn

graph of polling between API and clients

Current Solutions

Polling

Polling describes the mechanism used to retrieve data from an API: the client first needs to send a request to a server and the server responds by sending the requested data. Since it is not possible for the client to know when the API data is updated, it usually sends requests as often as possible to try to stick to reality and ends up using a lot of bandwidth and resources to receive the same data several times.

Polling Example

Let’s take a more in-depth look at the polling mechanism, and what happens when an application keeps its data current by using this method.

Response #1

The client requests data and is sent the whole data set requested.

 [
   { "name": "foo" },
   { "name": "bar" },
   { "name": "foo foo" },
   { "name": "bar bar" }
 ]
Response #2

The client makes another request. Nothing has change on the API, so the server responses by sending the same set of data as before.

 [
   { "name": "foo" },
   { "name": "bar" },
   { "name": "foo foo" },
   { "name": "bar bar" }
 ]
Response #3

The client makes another request and this time, there's new data available when the server responds.. Note how the whole set of data is again received even if only the third item has changed. While the volume of updated data is pretty small, it is enough to understand how much bandwidth is wasted by the polling mechanism.

 [
   { "name": "foo" },
   { "name": "bar" },
   { "name": "FOO BAR" },
   { "name": "bar bar" }
 ]
Streamdata.io solution to polling

Streamdata.io Solution

Data push with Server-Sent Events

Go beyond the traditional request/response mechanism and the limits that come with this approach. Streamdata.io uses Server-Sent-Events in order to provide its push feature. Instead of requesting data every time, the client subscribes to receive updates coming from the server in an automated fashion. That's streaming. No more polling.

Streamdata.io Solution

Incremental Update with JSON Patch

When new data is available on the API, instead of sending the whole set of data that includes new and unchanged fields, Streamdata.io performs differential computing. Using JSON Patch, only the updated data is then sent to the client, saving a lot of bandwidth in the process.

Response #1

After subscribing to the streaming API, the client is sent the whole data set requested.

{ 
  "id": "f81d4fae-7dec-11d0-a765-00a0c91e6bf6",
  "event": "snapshot",
  "data": [
    { "name": "foo" },
    { "name": "bar" },
    { "name": "foo foo" },
    { "name": "bar bar" }
  ] 
}
Response #2

The client doesn't have to request anything any longer. No new data is available, so nothing is sent to it.

//No changes detected 
Response #3

As new data becomes available via an API, a JSON Patch containing only the portion that has changed is sent to the client. Here, instead of receiving the four fields, the client receives instructions to update only the third element with the value FOO BAR."

{ 
    "id": "f81d4fae-7dec-11d0-a765-00a0c91e6bf6",
    "event": "delta",
    "data": [
    { "op":"replace", "path": "/3/name", "value": "FOO BAR" } 
  ]
}

Run Any API to See How Streamdata.io Works

See Streaming In Action Now!

Try streaming any JSON REST API within 30 sec

curl-v"https://proxy.streamdata.io/http:mysite.com/myJsonRestService?parameter1=[]&Parameter2=[]"