FreeCodeCamp and Learning Node: Juggling ASync

On to the next exercise in the FreeCodeCamp learnyounode sequence!  The processes are asynchronous, but we want to tame them so they get us our data in order.  Let’s look at the directions and hints:

JUGGLING ASYNC (Exercise 9 of 13)

  This problem is the same as the previous problem (HTTP COLLECT) in that you need to use http.get(). However, this time you will be provided with three URLs as the first three command-line arguments.
  You must collect the complete content provided to you by each of the URLs and print it to the console (stdout). You don’t need to print out the length, just the data as a String; one line per URL. The catch is that you must print them out in the same order as the URLs are provided to you as command-line arguments.
 ─────────────────────────────────────────────────────────────────────────────
 ## HINTS
  Don’t expect these three servers to play nicely! They are not going to give you complete responses in the order you hope, so you can’t naively just print the output as you get it because they will be out of order.
  You will need to queue the results and keep track of how many of the URLs have returned their entire contents. Only once you have them all, you can print the data to the console.
  Counting callbacks is one of the fundamental ways of managing async in Node. Rather than doing it yourself, you may find it more convenient to rely on a third-party library such as [async]( http://npm.im/async ) or [after]( http://npm.im/after ). But for this exercise, try and do it without any external helper library.

Okay, as always, We want to declare our variables first.  We’re going to need http, since we’re using http.get().  We’re also going to want bl, since we are using buffer list for the contents of the URLs, as we did in the previous exercise.  We’re also going to use the skeleton of the call to http.get() that we used in the last exercise:

var bl = require('bl');
var http = require('http');

http.get(addr, function callback(response) {

response.pipe(bl(function (err, data) {
if(err)
console.log(err);
}))

});

Okay, so if we run this now, we get back… a mess.  Remember, there are THREE inputs coming that we want to process one after the other – this suggests a loop.

So, let’s put in a loop:

for(var count = 0; count < process.argv.length; count++) {

http.get(process.argv[count], function callback(response) {

response.pipe(bl(function (err, data) {
if(err) {
console.log(err);

console.log(data.length);
console.log(data.toString());
}));

});
};

Umm…ewww.  No, that was no good.  I messed around with this for awhile, and still didn’t get what I wanted.  At one point, I came close – getting the results back, but not in order.  So, I did some web surfing, as suggested by FreeCodeCamp (you remember how they always say to use Read-Search-Ask, right?), and found this very helpful link on “Four ways of handling asynchronous operations in node.js“.  If you look about halfway down the page, you’ll see a section titled “Reading the files sequentially using callbacks”.

 

Now, here the poster is reading the contents of files, and not URLs.  He is also using fs.readdir.  But the idea is the same.

So, we create a function that contains our http.get() called function readURL().  We need to make it recursive, so it goes through the items provided to us by the arguments.  Keep in mind that, as always with Node, we start with the third element (element[2] since the array starts at 0).  So count’s initial value is 2.

In the response.pipe() statement, we will print the data out – as a string, as mentioned in the directions – and we add that statement.  Also, readURL() is recursive, so if it is successful, we want to call it again, only incrementing count by 1 so that it calls the next argument in process.argv.  If the next argument doesn’t exist, the process terminates and we’re finished.  So, now we have:

var bl = require('bl');
var http = require('http');
var count = 2;

function readURL(count) {

http.get(process.argv[count], function callback(response) {

response.pipe(bl(function (err, data) {
if(err) {
console.log(err);
} else {
console.log(data.toString());
readURL(count + 1);
}
}));

});
}

Let’s run it.

I bet you’re not surprised it doesn’t work, right?  Do you know why?

Because we created a function called readURL()…but we never called it.  We need to add that to the VERY bottom and try again.

 

It worked! And now we can move on to the next exercise.  Here’s the final program:

var bl = require('bl');
var http = require('http');
var count = 2;

function readURL(count) {

http.get(process.argv[count], function callback(response) {

response.pipe(bl(function (err, data) {
if(err) {
console.log(err);
} else {
console.log(data.toString());
readURL(count + 1);
}
}));

});
}

readURL(count);

See you next time!

Advertisements

One thought on “FreeCodeCamp and Learning Node: Juggling ASync

  1. Your final code seems going wrong for it won’t break out the loop. Here it is the update:

    “`js
    var http = require(‘http’);
    var bl = require(‘bl’);
    var count = 2;

    function parseURL(count) {
    http.get(process.argv[count], function callback(response) {
    response.pipe(bl(function (err, data) {
    if (err) {
    return console.error(err);
    }
    else {
    console.log(data.toString());
    if (count <= 3) {
    parseURL(count+1);
    }
    }
    }));
    });
    }

    parseURL(count);
    “`

    Thanks for your inspiration.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s