Use streamdata.io and Yahoo! Pipes to stream RSS Feeds

UPDATE June 29th – Yahoo! has finally given up with Y! Pipes, as of August 30th 2015, users will no longer be able to create new Pipes. The Pipes team will keep the infrastructure running until end of September 30th 2015 in a read-only mode (Y! Pipes End-of-Life Announcement).
Check out this blog post to find alternatives.
Here is a quick and easy 2-step tutorial to get your favorite RSS Feeds in streaming using Yahoo! Pipes and streamdata.io.

For those of you who do not know Yahoo! Pipes yet, it is a simple yet powerful tool that helps you mashup content from the web. It provides a drag and drop editor that helps you to create logic to fetch your content from different types of data sources (RSS, XML, CSV and more), aggregate and/or manipulate content (filtering, sorting) and finally let you grab the output in different formats (RSS, JSON and other formats). See more details here. We will be using Yahoo! Pipes to aggregate two different RSS feeds and streamdata.io to push aggregated content down to a Javascript client application. Using streamdata.io in this context will prevent a client app from frequently polling the source to know if new content is available. Streamdata.io will poll the source and push updated content as soon as it is available. If you want to know more about streamdata.io read our How it works page.

1. Create your Yahoo! Pipe
It is quite an easy thing to do:
  1. Go to https://pipes.yahoo.com/pipes/. Click on Create Pipe button, it will open the Yahoo Pipe editor.
  2. Drag and drop two different Fetch Feed modules from the list under Sources section.
  3. Enter the URL of your favorite RSS feed in each of your Fetch Feed modules.
  4. Drag and drop a Union operator that you will find under the Operators section to aggregate both feeds.
  5. Connect each Fetch Feed module to your Union operator.
  6. Finally, connect the Union operator to the Pipe Output.
  7. Then, click Save your pipe button.
Result in Yahoo! Pipes Editor should look like something like this:

Capture-d’écran-2015-04-27-à-10.49.58

To test the result, click on Run Pipe (link is located at the very top of the Editor), you should see a preview of your aggregated content.
You are done with the first step!
2. Connect streamdata.io to your Yahoo! Pipe
  1. As streamdata.io only accepts JSON formatted data as input, you will have to convert RSS to JSON. Thanks to Yahoo! Pipes it’s straightforward: you just have to click on Get as JSON link once you have tested your Pipe.
  2. If you are not a streamdata.io user yet, you can quickly create an account here. You will get your public/private key to connect your client app to our service.
  3. Once registered, use our JavaScript sample application (using Angular or JQuery UI Framework) available on GitHub. Run either the Angular or JQuery application by launching the index.html in your favorite browser (FYI, it is not working with IE yet but wait if IE is your favorite browser please do some self-analysis first :-) )
  4. Copy/paste the link of your Yahoo! Pipe in JSON format, your public/private key and click Connect.
You should now receive a first snapshot of your aggregated content followed by JSON-Patch updates whenever content is changing!

Capture-d’écran-2015-04-27-à-11.36.21

 

Share it :
0000

Give it a try!

Try streaming any JSON REST API within 30 sec
curl -v "https://proxy.streamdata.io/http://mysite.com/myJsonRestService?param1=[]&param2=[]"

Leave a Reply

Your email address will not be published. Required fields are marked *